IEEE Transactions on Audio Speech and Language Processing · 2011 · 66 citations · 42 references
ComputingComputer ScienceComputer VisionPsychoacousticsVisual CommunicationSpeech PerceptionSpeech RecognitionAudio-visual Speech RecognitionBackground Noise EffectsConfidence MeasuresExperimental Results InterpretationSpeech Activity DetectionSpeech Recognition Accuracy
The integration of audio and visual information improves speech recognition performance, specially in the presence of noise. In these circumstances it is necessary to introduce audio and visual weights to control the contribution of each modality to the recognition task. We present a method to set the value of the weights associated to each stream according to their reliability for speech recognition, allowing them to change with time and adapt to different noise and working conditions. Our dynamic weights are derived from several measures of the stream reliability, some specific to speech processing and others inherent to any classification task, and take into account the special role of silence detection in the definition of audio and visual weights. In this paper, we propose a new confidence measure, compare it to existing ones, and point out the importance of the correct detection of silence utterances in the definition of the weighting system. Experimental results support our main contribution: the inclusion of a voice activity detector in the weighting scheme improves speech recognition over different system architectures and confidence measures, leading to an increase in performance more relevant than any difference between the proposed confidence measures.
42
Neural networks for pattern recognition
Choice Reviews Online · 1994 · 18.7K citations
Josef Kittler, M. Hatef, Robert P. W. Duin et al. · IEEE Transactions on Pattern Analysis and Machine Intelligence · 1998 · 5.3K citations
An introduction to hidden Markov models
L. R. Rabiner, Biing‐Hwang Juang · IEEE ASSP Magazine · 1986 · 4.8K citations
Jorge J. Morè, D. C. Sorensen · SIAM Journal on Scientific and Statistical Computing · 1983 · 1.4K citations · Full text