Interspeech 2022 · 2022 · 23 citations · 33 references
ComputingComputer ScienceComputer VisionNatural Language ProcessingVisual CommunicationMultimodal CorporaMultimodal InteractionSpeech RecognitionSpoken CorporaAnnotation Of Video DataAudio-visual CorporaAudio-visual Speech RecognitionData AugmentationEnd-to-end Speech RecognitionMicrophonesVideo Corpus Analysis
<p>In this paper, we present the updated Audio-Visual Speech Recognition (AVSR) corpus of MISP2021 challenge, a large-scale audio-visual Chinese conversational corpus consisting of 141h audio and video data collected by far/middle/near microphones and far/middle cameras in 34 real-home TV rooms. To our best knowledge, our corpus is the first distant multi-microphone conversational Chinese audio-visual corpus and the first large vocabulary continuous Chinese lip-reading dataset in the adverse home-tv scenario. Moreover, we make a deep analysis of the corpus and conduct a comprehensive ablation study of all audio and video data in the audio-only/video-only/audiovisual systems. Error analysis shows video modality supplement acoustic information degraded by noise to reduce deletion errors and provide discriminative information in overlapping speech to reduce substitution errors. Finally, we also design a set of experiments such as frontend, data augmentation and end-to-end models for providing the direction of potential future work. The corpus and the code are released to promote the research not only in speech area but also for the computer vision area and cross-disciplinary research.</p>
33
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
Hearing lips and seeing voices
Harry McGurk, John MacDonald · Nature · 1976 · 6.1K citations
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
Daniel Park, William Chan, Yu Zhang et al. · 2019 · 3.5K citations · Full text
Conformer: Convolution-augmented Transformer for Speech Recognition
Anmol Gulati, James Qin, Chung‐Cheng Chiu et al. · 2020 · 2.7K citations