2001 · 35 citations · 5 references
Language StudiesLinguistics3D ModelingComputer VisionPhoneticsArticulatory PhoneticsMultimodal InteractionSpeech InteractionSpeech RecognitionSpeech SynthesisAudio-visual Speech RecognitionAudiovisual PerceptionModel RegularizationQuantizationQuantization NoiseRegularization
We present a linear three-dimensional modeling paradigm for lips and face, that captures the audiovisual speech activity of a given speaker by only six parameters. Our articulatory models are constructed from real data (front and profile images), using a linear component analysis of about 200 3D coordinates of fleshpoints on the subject's face and lips. Compared to a raw component analysis, our construction approach leads to somewhat more comparable relations across subjects: by construction, the six parameters have a clear phonetic/articulatory interpretation. We use such a speaker’s specific articulatory model to regularize MPEG-4 facial articulation parameters (FAP) and show that this regularization process can drastically reduce bandwidth, noise and quantization artifacts. We then present how analysis-by-synthesis techniques using the speaker-specific model allows the tracking of facial movements. Finally, the results of this tracking scheme have been used to develop a text-to-audiovisual speech system. 1
5
Numerical Model of Coarticulation
Sven Öhman · The Journal of the Acoustical Society of America · 1967 · 294 citations
Face and 2-D mesh animation in MPEG-4
A. Murat Tekalp, Jörn Östermann · Signal Processing Image Communication · 2000 · 201 citations
A three-dimensional linear articulatory model based on MRI data
Pierre Badin, Gérard Bailly, Monica Raybaudi et al. · 1998 · 54 citations
Medicine, Anatomy, 3D Modeling +11