1999 · 200 citations · 0 references
ComputingComputer ScienceCorpus LinguisticsInformation RetrievalNatural Language ProcessingAudio RecordingEvaluation Of Retrieval SystemsRetrieval ModelsSpeech RecognitionAudio RetrievalRetrieval EvaluationSpeech Recognition EvaluationSpeech RetrievalText RetrievalTrec EvaluationTrec Evaluation Framework
Spoken Document Retrieval searches spoken audio using automatic speech recognition and information retrieval, and the TREC SDR Track has provided infrastructure and a forum for advancing this technology. The paper reports on three years of TREC work designing and implementing evaluations of Spoken Document Retrieval technology in broadcast news, and outlines plans and future directions for the 2000 track. The authors designed and implemented a series of SDR evaluations, presenting their design and summarizing the results. The SDR Track proved successful, demonstrating that existing technologies can effectively retrieve from realistic audio collections and that the system can be objectively evaluated, with results summarized for each evaluation.
This paper describes work within the NIST Text REtrieval Conference (TREC) over the last three years in designing and implementing evaluations of Spoken Document Retrieval (SDR) technology within a broadcast news domain. SDR involves the search and retrieval of excerpts from spoken audio recordings using a combination of automatic speech recognition and information retrieval technologies. The TREC SDR Track has provided an infrastructure for the development and evaluation of SDR technology and a common forum for the exchange of knowledge between the speech recognition and information retrieval research communities. The SDR Track can be declared a success in that it has provided objective, demonstrable proof that this technology can be successfully applied to realistic audio collections using a combination of existing technologies and that it can be objectively evaluated. The design and implementation of each of the SDR evaluations are presented and the results are summarized. Plans for the 2000 TREC SDR Track are presented and thoughts about how the track might evolve are discussed.