Max-pooling convolutional neural networks for vision-based hand gesture recognition
Jawad Nagia b, Frederick Ducatellea b, Gianni A. Dia b, Dan Claudiu Ciresana b, Ueli Meiera b, Alessandro Giustia b, Farrukh Nagic d, Jürgen Schmidhubera b, Luca Maria Gambardellaa b
- aDalle Molle Institute for Artificial Intelligence (IDSIA), Dalle Molle Institute for Artificial Intelligence (IDSIA), University of Lugano, Manno Lugano, Switzerland
- bDalle Molle Institute for Artificial Intelligence (IDSIA), University of Lugano & SUPSI, Manno-Lugano, Switzerland
- cDepartment of Mechanical Engineering, University Tenaga National, Putrajaya, Selangor Darul Ehsan, Malaysia
- dDepartment of Mechanical Engineering, University Tenaga Nasional Putrajaya, Selangor, Malaysia
Papers open inside SciRah from the original publisher or repository; SciRah never hosts them.
Abstract
Automatic recognition of gestures using computer vision is important for many real-world applications such as sign language recognition and human-robot interaction (HRI). Our goal is a real-time hand gesture-based HRI interface for mobile robots. We use a state-of-the-art big and deep neural network (NN) combining convolution and max-pooling (MPCNN) for supervised feature learning and classification of hand gestures given by humans to mobile robots using colored gloves. The hand contour is retrieved by color segmentation, then smoothened by morphological image processing which eliminates noisy edges. Our big and deep MPCNN classifies 6 gesture classes with 96% accuracy, nearly three times better than the nearest competitor. Experiments with mobile robots using an ARM 11 533MHz processor achieve real-time gesture recognition performance.