Telepresence Robot with Image-Based Face Tracking and 3D Perception with Human Gesture Interface Using Kinect Sensor

Abstract

This paper presents a Telepresence Mobile Robot using a Kinect sensor as the main perception/interface device. Firstly, using the Kinect camera (Webcam) and image processing techniques, it is possible to detect a human face, allowing the robot to track the face, getting closer of a person, moving forward and rotating to get a better position to interact with him/her. Besides that, it is possible to recognize hand gestures using the Kinect 3D sensor, when correctly positioned related to the person. The proposed gesture recognition method tracks the hands positions and movements, when moving it forward towards the robot, and then recognizing a set of predefined gestures/commands. Finally, the 3D perception provided by the Kinect also allows us to detect obstacles, avoiding collisions and helping to move safely and to search for someone in the environment. Practical experiments are presented demonstrating the obtained results in the searching, tracking of faces and obot positioning related to the user, and also, in the gesture recognition.