Local normal binary patterns for 3D facial action unit detection

Abstract

This paper proposes a new feature descriptor, local normal binary patterns (LNBPs), which is exploited for detection of facial action units (AUs). After LNBPs have been employed to form descriptor vectors, which capture the detailed shape of the action, feature selection is performed via a Gentle-Boost (GB) algorithm, and support vector machines (SVMs) are trained to detect each AU. This process was tested on the Bosphorus database, alongside the same test using 3D local binary pattern (3DLBP) descriptors which apply the LBP operator to the depth map of the face. LNBP descriptors were demonstrated to outperform 3DLBPs in detection of many individual AUs. Finally, feature fusion was used to combine the benefits of the 3DLBPs and each of the LNBP descriptors, with the best result achieving a mean ROC AuC of 96.35.