Automatic analysis of facial expressions: the state of the art

TLDR

Humans can effortlessly detect and interpret facial expressions, yet automating this process is difficult because it requires accurate face detection, expression extraction, and emotion classification. The paper seeks to create a real‑time, accurate system for these tasks to enable human‑like interaction and to provide guidance for future development of automatic facial expression analyzers. The authors review prior research on face detection, expression extraction, and emotion classification.

Abstract

Humans detect and interpret faces and facial expressions in a scene with little or no effort. Still, development of an automated system that accomplishes this task is rather difficult. There are several related problems: detection of an image segment as a face, extraction of the facial expression information, and classification of the expression (e.g., in emotion categories). A system that performs these operations accurately and in real time would form a big step in achieving a human-like interaction between man and machine. The paper surveys the past work in solving these problems. The capability of the human visual system with respect to these problems is discussed, too. It is meant to serve as an ultimate goal and a guide for determining recommendations for development of an automatic facial expression analyzer.