Skeleton and visual tracking fusion for human following task of service robots

Abstract

In this paper, we propose a novel method to overcome some of the weaknesses of typical skeleton trackers, which use depth data for the task of human following in robotics. We used our service robot, Sepanta, for evaluations. Skeleton trackers such as Microsoft Kinect SDK (KST) or OpenNI, NITE extension Skeleton tracker (NST) lose the track of skeletons in occlusion or target lost situations and cannot recover from that. On the other hand, there are other visual approaches that use only a monocular camera for object tracking such as the state of the art method, OpenTLD. In our novel approach we combine typical skeleton tracker with state of the art OpenTLD visual tracker using Kalman filter. We track the desired person with skeleton tracker, then select a region of interest autonomously and perform TLD tracking to learn and track the target person. If there is a problem with the skeleton tracker, visual tracker will perform the tracking task and if the visual tracker is locked in non-human regions, the skeleton tracker will take over the tracking.