Rapid perceptual integration of facial expression and emotional body language

TLDR

In natural contexts, faces are typically perceived as part of the whole body, and both facial and bodily cues jointly convey emotional states. The study used photographs of fearful and angry faces and bodies to create matched or mismatched compound images, and recorded scalp electrical activity while participants attended to the face and judged its emotion. Observers’ judgments of facial expression were strongly biased by the body’s emotion, especially when face and body conflicted, and an early occipital P1 enhancement at 115 ms revealed a rapid neural mechanism sensitive to face–body agreement even when the body was unattended.

Abstract

In our natural world, a face is usually encountered not as an isolated object but as an integrated part of a whole body. The face and the body both normally contribute in conveying the emotional state of the individual. Here we show that observers judging a facial expression are strongly influenced by emotional body language. Photographs of fearful and angry faces and bodies were used to create face-body compound images, with either matched or mismatched emotional expressions. When face and body convey conflicting emotional information, judgment of facial expression is hampered and becomes biased toward the emotion expressed by the body. Electrical brain activity was recorded from the scalp while subjects attended to the face and judged its emotional expression. An enhancement of the occipital P1 component as early as 115 ms after presentation onset points to the existence of a rapid neural mechanism sensitive to the degree of agreement between simultaneously presented facial and bodily emotional expressions, even when the latter are unattended.