Gesture–speech physics: The biomechanical basis for the emergence of gesture–speech synchrony.

Abstract

The phenomenon of gesture-speech synchrony involves tight coupling of prosodic contrasts in gesture movement (e.g., peak velocity) and speech (e.g., peaks in fundamental frequency; F0). Gesture-speech synchrony has been understood as completely governed by sophisticated neural-cognitive mechanisms. However, gesture-speech synchrony may have its original basis in the resonating forces that travel through the body. In the current preregistered study, movements with high physical impact affected phonation in line with gesture-speech synchrony as observed in natural contexts. Rhythmic beating of the arms entrained phonation acoustics (F0 and the amplitude envelope). Such effects were absent for a condition with low-impetus movements (wrist movements) and a condition without movement. Further, movement-phonation synchrony was more pronounced when participants were standing as opposed to sitting, indicating a mediating role for postural stability. We conclude that gesture-speech synchrony has a biomechanical basis, which will have implications for our cognitive, ontogenetic, and phylogenetic understanding of multimodal language. (PsycINFO Database Record (c) 2020 APA, all rights reserved).