A Visual Cortical Network for Deriving Phonological Information from Intelligible Lip Movements

Anne Hauswald, Chrysa Lithari, Olivier Collignon, Elisa Leonardelli, Nathan Weisz

Current Biology · 2018 · 74 citations · 39 references

DOIFull text

Open access

Concepts

Abstract

Successful lip-reading requires a mapping from visual to phonological information [1Lazard D.S. Giraud A.L. Faster phonological processing and right occipito-temporal coupling in deaf adults signal poor cochlear implant outcome.Nat. Commun. 2017; 8: 14872Crossref PubMed Scopus (24) Google Scholar]. Recently, visual and motor cortices have been implicated in tracking lip movements (e.g., [2Park H. Kayser C. Thut G. Gross J. Lip movements entrain the observers’ low-frequency brain oscillations to facilitate speech intelligibility.eLife. 2016; 5: e14521Crossref PubMed Scopus (75) Google Scholar]). It remains unclear, however, whether visuo-phonological mapping occurs already at the level of the visual cortex–that is, whether this structure tracks the acoustic signal in a functionally relevant manner. To elucidate this, we investigated how the cortex tracks (i.e., entrains to) absent acoustic speech signals carried by silent lip movements. Crucially, we contrasted the entrainment to unheard forward (intelligible) and backward (unintelligible) acoustic speech. We observed that the visual cortex exhibited stronger entrainment to the unheard forward acoustic speech envelope compared to the unheard backward acoustic speech envelope. Supporting the notion of a visuo-phonological mapping process, this forward-backward difference of occipital entrainment was not present for actually observed lip movements. Importantly, the respective occipital region received more top-down input, especially from left premotor, primary motor, and somatosensory regions and, to a lesser extent, also from posterior temporal cortex. Strikingly, across participants, the extent of top-down modulation of the visual cortex stemming from these regions partially correlated with the strength of entrainment to absent acoustic forward speech envelope, but not to present forward lip movements. Our findings demonstrate that a distributed cortical network, including key dorsal stream auditory regions [3Hickok G. Poeppel D. The cortical organization of speech processing.Nat. Rev. Neurosci. 2007; 8: 393-402Crossref PubMed Scopus (3240) Google Scholar, 4Rauschecker J.P. Ventral and dorsal streams in the evolution of speech and language.Front. Evol. Neurosci. 2012; 4: 7Crossref PubMed Scopus (94) Google Scholar, 5Rauschecker J.P. Cortical processing of complex sounds.Curr. Opin. Neurobiol. 1998; 8: 516-521Crossref PubMed Scopus (419) Google Scholar], influences how the visual cortex shows sensitivity to the intelligibility of speech while tracking silent lip movements.

References

39