Voice/mouth synthesis and tactual/visual perception of /pa, ba, ma/

Abstract

The onset of voicing relative to onset of visible articulation is a potential cue for lipreaders to recognize nasal/stop consonants. The bilabial consonants /p, b, m/ were selected for study because of their high visibility and ease of optical synthesis. A phototransistor behind the teeth and an external microphone indicated mouth opening and voice onset, respectively, to measure relative timing of the two cues in natural speech. Average voicing leads for ten adults were the following: /ma/, +66 ms; /ba/, −1 ms; /pa/, −58 ms (/ma–ba/ articulatory boundary, +28 ms; /ba–pa/ articulatory boundary, −18 ms). To test relative timing of voice onset and mouth opening as a potential perceptual cue, we produced synthetic ’’syllables’’: The face substitute was an oscilloscope pattern, forming a mouth shape for 250, 400, or 550 ms; the voice substitute was a vowel-like buzz delivered through a hand-held vibrator. Buzz accompanied mouth opening with lead times of from +200 to −200 ms (40-ms steps), and ended simultaneously with mouth closure. Normal-hearing observers labeled each combination ’’ma,’’ ’’ba,’’ or ’’pa.’’ Practiced lipreaders with knowledge of the onset cue subdivided the continuum with mean boundaries at +82 ms (/ma–ba/) and −87 ms (/ba–pa/). Individual differences occurred in sharpness and location of labeling boundaries. Because articulatory and perceptual boundaries are not coincident, this combined-mode cue appears insufficient for consonant recognition unless performance can be improved through extensive training.