2002 · 53 citations · 15 references
ComputingComputer ScienceComputer VisionPhoneticsPhonologyImage CompressionSpeech Intelligibility EnhancementSpeech RecognitionAudio ClassificationAudio-visual Speech RecognitionAudio-visual SystemsClassification AccuracyLinear Discriminant AnalysisMaximum Likelihood ClassificationSpeaker-independent Speech Recognition
We propose a three-stage pixel based visual front end for automatic speechreading (lipreading) that results in improved recognition performance of spoken words or phonemes. The proposed algorithm is a cascade of three transforms applied to a three-dimensional video region of interest that contains the speaker's mouth area. The first stage is a typical image compression transform that achieves a high "energy", reduced-dimensionality representation of the video data. The second stage is a linear discriminant analysis based data projection, which is applied to a concatenation of a small number of consecutive image transformed video data. The third stage is a data rotation by means of a maximum likelihood linear transform. Such a transform optimizes the likelihood of the observed data under the assumption of their class conditional Gaussian distribution with diagonal covariance. We apply the algorithm to visual-only 52-class phonetic and 27-class visemic classification on a 162-subject, 7-hour long, large vocabulary, continuous speech audio-visual dataset. We demonstrate significant classification accuracy gains by each added stage of the proposed algorithm, which, when combined, can reach up to 27% improvement. Overall, we achieve a 49% (38%) visual-only frame level phonetic classification accuracy with (without) use of test set phone boundaries. In addition, we report improved audio-visual phonetic classification over the use of a single-stage image transform visual front end.
15
Advances in neural information processing systems 7
Computers & Mathematics with Applications · 1996 · 14.4K citations
Numerical Recipes--The Art of Scientific Computing.
Frederick N. Fritsch, William H. Press, Brian P. Flannery et al. · Mathematics of Computation · 1988 · 4.9K citations
Numerical Recipes in C: The Art of Scientific Computing
Mary C. Seiler, Fritz A. Seiler · Risk Analysis · 1989 · 1.4K citations
"Eigenlips" for robust speech recognition
Christoph Bregler, Yochai Konig · 2002 · 314 citations