Comprehensive database for facial expression analysis

TLDR

Facial expression analysis has seen extensive method development, but limited datasets hinder generalizability. The study outlines the problem space of facial expression analysis, covering description levels, transitions, elicitation, data reliability, subject differences, head orientation, scene complexity, image characteristics, and links to non‑verbal behavior. The authors introduce the CMU‑Pittsburgh AU‑Coded Face Expression Image Database, comprising 2105 image sequences from 182 diverse adult subjects performing multiple primary FACS action units. The database is the most comprehensive testbed to date for comparative studies of facial expression analysis.

Abstract

Within the past decade, significant effort has occurred in developing methods of facial expression analysis. Because most investigators have used relatively limited data sets, the generalizability of these various methods remains unknown. We describe the problem space for facial expression analysis, which includes level of description, transitions among expressions, eliciting conditions, reliability and validity of training and test data, individual differences in subjects, head orientation and scene complexity image characteristics, and relation to non-verbal behavior. We then present the CMU-Pittsburgh AU-Coded Face Expression Image Database, which currently includes 2105 digitized image sequences from 182 adult subjects of varying ethnicity, performing multiple tokens of most primary FACS action units. This database is the most comprehensive testbed to date for comparative studies of facial expression analysis.