Video parsing based on head tracking and face recognition

Abstract

In this paper, we describe a fully automatic video retrieval prototype system that uses an image or a video sequence of an interested identity as probe. The system is based on face vision techniques including face detection and tracking, face alignment and recognition. Given a film or TV sitcom, first face trajectories are extracted in video by head tracking that decompose the video into segments corresponding to certain identity, then frames containing faces of higher quality are selected and normalized according to face alignment results, and finally different segments are associated by face recognition. Experiments are carried out on news video, feature length film video and TV sitcom to show its effectiveness. Potential usage of our system includes intelligent DVD/VCD browsing, video database retrieval, meeting record browsing, etc.