Behind the palm: Hand gesture recognition through measuring skin deformation on back of hand by using optical sensors

Abstract

We propose a system consisting of a wearable device equipped with photo-reflective sensors arranged in an array. Hand gestures are recognized by measuring the skin deformation of the back of the hand. Since the muscles and bones on the back of the hand are linked to the fingers, finger movement can be clearly observed. Skin deformation is measured using several photo-reflective sensors. Skin deformation can be determined by measuring the distance between the device and skin with these sensors. The system estimates hand gestures with a support vector machine using the sensor data. Since this system simultaneously records the hand shape using Leap Motion in the learning phase, a user can freely register gestures. The system further displays a reconstructed digital hand as a gesture-recognition result.