Exploring Facial Expression Recognition through Semi-Supervised Pre-training and Temporal Modeling

Abstract

Facial Expression Recognition (FER) plays a crucial role in computer vision and finds extensive applications across various fields. This paper aims to present our approach for the 6th Affective Behavior Analysis in-the-Wild (ABAW) competition, scheduled to be held at CVPR2024. In the facial expression recognition task, the limited size of the FER dataset poses a challenge to the expression recognition model’s generalization ability, resulting in subpar recognition performance. To address this problem, we employ a Semi-supervised learning technique to generate expression category pseudo labels for unlabeled face data. At the same time, we uniformly sampled the labeled facial expression samples and implemented a debiased feedback learning strategy to address the problem of category imbalance in the dataset and the possible data bias in semi-supervised learning. Moreover, to further compensate for the limitation and bias of features obtained only from static images, we introduced a Temporal Encoder to capture temporal relationships between neighbouring expression image features. In the 6th ABAW competition, our method achieved the third place in the official test set, a result that fully demonstrates the effectiveness and competitiveness of our proposed method.