Confidence-Weighted Local Expression Predictions for Occlusion Handling\n in Expression Recognition and Action Unit detection

Abstract

Fully-Automatic Facial Expression Recognition (FER) from still images is a\nchallenging task as it involves handling large interpersonal morphological\ndifferences, and as partial occlusions can occasionally happen. Furthermore,\nlabelling expressions is a time-consuming process that is prone to\nsubjectivity, thus the variability may not be fully covered by the training\ndata. In this work, we propose to train Random Forests upon spatially defined\nlocal subspaces of the face. The output local predictions form a categorical\nexpression-driven high-level representation that we call Local Expression\nPredictions (LEPs). LEPs can be combined to describe categorical facial\nexpressions as well as Action Units (AUs). Furthermore, LEPs can be weighted by\nconfidence scores provided by an autoencoder network. Such network is trained\nto locally capture the manifold of the non-occluded training data in a\nhierarchical way. Extensive experiments show that the proposed LEP\nrepresentation yields high descriptive power for categorical expressions and AU\noccurrence prediction, and leads to interesting perspectives towards the design\nof occlusion-robust and confidence-aware FER systems.\n