Social Signal Processing [Exploratory DSP]

TLDR

Social signaling in face‑to‑face communication conveys context, attitudes, and nonlinguistic cues that shape group understanding and cohesion. The paper explores challenges in exploratory processing of social signals and presents tools that can predict human behavior, sometimes surpassing expert human performance. These tools can automatically insert people into key conversations, initiate interactions to strengthen social networks, and reinforce family ties. Such tools enable computer and communication systems to support social and organizational roles rather than treating individuals as isolated entities.

Abstract

Face-to-face communication conveys social context as well as words. It is this social signaling that allows new information to be smoothly integrated into a shared, group-wide understanding. Social signaling includes signals of interest, determination, friendliness, boredom, and other "attitudes" toward a social situation. Psychologists speculate that social signaling may have evolved as a way to establish hierarchy and group cohesion because social signaling functions as a subconscious discussion about relationships, resources, risks, and rewards. In many situations the nonlinguistic signals that serve as the basis for this social discussion are just as important as conscious content for determining human behavior. In what follows we discuss challenges in exploratory processing of social signals and tools that allow us to predict human behavior and sometimes exceed even expert human capabilities. These tools potentially permit computer and communications systems to support social and organizational roles instead of viewing the individual as an isolated entity. Example applications include automatically patching people into socially important conversations, instigating conversations among people in order to build a more solid social network, and reinforcing family ties.