Automatic Extraction of Lip Feature Points

Abstract

We present a novel algorithm for the robust and reliable automatic extraction of lip feature points for speechreading. The algorithm uses a combination of colour information in the image data and knowledge about the structure of the mouth area to find certain feature points on the inner lip contour. A new confidence measure quantifying how well the feature extraction process worked is introduced. A parameter set describing the shape of the mouth is derived from the positions of the feature points. Using a stereo camera system, measurements are in 3D. Such a 3D parameter set is of great value for automatic speech-reading systems.