Max-pooling convolutional neural networks for vision-based hand gesture recognition

TLDR

Automatic gesture recognition via computer vision is crucial for applications such as sign language interpretation and human‑robot interaction. The authors aim to provide a real‑time hand‑gesture HRI interface for mobile robots. They employ a large, deep max‑pooling convolutional neural network that learns features and classifies gestures from colored gloves, with hand contours extracted by color segmentation and cleaned by morphological processing. The network attains 96 % accuracy on six gesture classes—roughly three times better than the nearest competitor—and operates in real time on an ARM 11 533 MHz processor during mobile‑robot experiments.

Abstract

Automatic recognition of gestures using computer vision is important for many real-world applications such as sign language recognition and human-robot interaction (HRI). Our goal is a real-time hand gesture-based HRI interface for mobile robots. We use a state-of-the-art big and deep neural network (NN) combining convolution and max-pooling (MPCNN) for supervised feature learning and classification of hand gestures given by humans to mobile robots using colored gloves. The hand contour is retrieved by color segmentation, then smoothened by morphological image processing which eliminates noisy edges. Our big and deep MPCNN classifies 6 gesture classes with 96% accuracy, nearly three times better than the nearest competitor. Experiments with mobile robots using an ARM 11 533MHz processor achieve real-time gesture recognition performance.