Real time head pose tracking from multiple cameras with a generic model

Abstract

We present a robust approach to real-time 3D head pose tracking using multiple cameras with unknown camera placements. Many important applications do not want prior multi-camera calibration. We exploit a generic face model to overcome the difficulties due to the lack of prior knowledge of camera placement and the severe head appearance difference across cameras. We propose a fast drift-free solution based on feature point tracking using reference frames of high confidence over the temporal and spatial domains. Our algorithm tracks feature points from Harris feature detector, but not necessarily points of face landmarks. The relative camera placement is refined progressively at the same time as the user's head pose is resolved. Compared to single camera tracking, the use of multiple cameras increases the reliability of tracking by covering a wider range of poses as well as providing more accurate head pose estimation. We have tested the algorithm on many subjects in a variety of environments. A live demonstration will be shown at the conference.