Facial Expression Recognition by Multi-Scale CNN with Regularized Center Loss

Abstract

Facial Expression Recognition (FER) has attracted considerable attention due to its potential applications in computer vision. Recently, convolutional neural network (CNN) has shown excellent performance on FER. However, most established deeper, wider and more complex network structures trained by small facial expression training dataset have a risk of overfitting. Moreover, most existing CNN models utilize the softmax loss as a supervision signal to penalize the deviation of classification, which enhances inter-class separation, yet intra-class compactness is not taken into consideration. In this paper, we propose a novel multi-scale CNN integrated with an attention-based learning layer (AMSCNN) for robust facial expression recognition. The attention-based learning layer is designed to automatically learn the importance of different receptive fields in the face during training. Moreover, the multi-scale CNN is optimized by the proposed Regularized Center Loss (RCL). Regularized center loss learns a center for deep features of each class and penalizes the distance between deep features and corresponding center, aiming to strengthen the discriminability of different facial expression. Extensive experiments conducted on two popular human FER benchmarks (CK+ and Oulu-CASIA dataset) demonstrated the effective of our proposed AMSCNN, and it obtained competitive results compared to the state-of-the-art.