Tracked Speech-To-Text Display

Abstract

Deaf and Hard of Hearing (DHH) students are under-served and under-represented in education in part because they miss spoken classroom information, even with aural-to-visual accommodations, such as a real-time speech to text Display (SD). Most SD systems utilize a trained typist to transcribe the speech into text (speech-to-text) onto a display. Still, these students encounter significant but subtle barriers in following speech-to-text displays, especially when detailed visuals are used or when the speaker is fast or uses uncommon words. Hearing students can simultaneously watch the visuals and listen to the spoken explanation, while DHH students constantly look away from the SD to search and observe details in the classroom visuals. As a result, they spend less time watching the visuals and gain less information than their hearing peers. They can also fall behind in reading the speech-text.