Joint Action Unit localisation and intensity estimation through heatmap\n regression

Abstract

This paper proposes a supervised learning approach to jointly perform facial\nAction Unit (AU) localisation and intensity estimation. Contrary to previous\nworks that try to learn an unsupervised representation of the Action Unit\nregions, we propose to directly and jointly estimate all AU intensities through\nheatmap regression, along with the location in the face where they cause\nvisible changes. Our approach aims to learn a pixel-wise regression function\nreturning a score per AU, which indicates an AU intensity at a given spatial\nlocation. Heatmap regression then generates an image, or channel, per AU, in\nwhich each pixel indicates the corresponding AU intensity. To generate the\nground-truth heatmaps for a target AU, the facial landmarks are first\nestimated, and a 2D Gaussian is drawn around the points where the AU is known\nto cause changes. The amplitude and size of the Gaussian is determined by the\nintensity of the AU. We show that using a single Hourglass network suffices to\nattain new state of the art results, demonstrating the effectiveness of such a\nsimple approach. The use of heatmap regression allows learning of a shared\nrepresentation between AUs without the need to rely on latent representations,\nas these are implicitly learned from the data. We validate the proposed\napproach on the BP4D dataset, showing a modest improvement on recent, complex,\ntechniques, as well as robustness against misalignment errors. Code for testing\nand models will be available to download from\nhttps://github.com/ESanchezLozano/Action-Units-Heatmaps.\n