Towards robust lip tracking

Abstract

An algorithm for the automatic extraction of speakers lips in video sequences is presented. Our goal is to extract minimum face feature parameters, vital for audio-visual communication, in adverse conditions and at a very low bit rate coding. Our method uses spatial (region and contour) and temporal (similarity function) information from luminance and hue components. A new literal inverse of the active contours stiffness matrix is introduced. This ensures a fast and accurate convergence of active contours towards lip boundaries. The use of the Kanade-Lucas tracking algorithm with our point extraction method leads to an automatic, fast and robust initialization of Snakes. The significant robustness enhancement and related computational cost decrease allow processing approaching real time.