2020 · 53 citations · 30 references
Language StudiesCommunicationLinguisticsInterpersonal CommunicationNonverbal CommunicationPsycholinguisticsConversation AnalysisMultimodal CorporaSpeech InteractionConversationsGesture-movement CorrespondenceGesture-speech RelationshipHand GesturesLanguage Processing ModelsPredictive Processing In LanguageResponse TimeSemantic Information
In face-to-face conversation, recipients might use the bodily movements of the speaker (e.g. gestures) to facilitate language processing. It has been suggested that one way through which this facilitation may happen is prediction. However, for this to be possible, gestures would need to precede speech, and it is unclear whether this is true during natural conversation. In a corpus of Dutch conversations, we annotated hand gestures that represent semantic information and occurred during questions, and the word(s) which corresponded most closely to the gesturally depicted meaning. Thus, we tested whether representational gestures temporally precede their lexical affiliates. Further, to see whether preceding gestures may indeed facilitate language processing, we asked whether the gesture-speech asynchrony predicts the response time to the question the gesture is part of. Gestures and their strokes (most meaningful movement component) indeed preceded the corresponding lexical information, thus demonstrating their predictive potential. However, while questions with gestures got faster responses than questions without, there was no evidence that questions with larger gesture-speech asynchronies get faster responses. These results suggest that gestures indeed have the potential to facilitate predictive language processing, but further analyses on larger datasets are needed to test for links between asynchrony and processing advantages.
30
R: A Language and Environment for Statistical Computing
R Core Team · 2000 · 353K citations · Full text
Natural Sciences, Computer Science, Statistical Computing +10
Fitting Linear Mixed-Effects Models Using <b>lme4</b>
Douglas M. Bates, Martin Mächler, Benjamin M. Bolker et al. · Journal of Statistical Software · 2015 · 85.6K citations · Full text
The Measurement of Observer Agreement for Categorical Data
J. Richard Landis, Gary G. Koch · Biometrics · 1977 · 78.9K citations · Full text