2023 · 17 citations · 17 references
ComputingComputer ScienceComputer VisionMachine LearningNatural Language ProcessingAction RecognitionDeep LearningEye ContactMultimodal InteractionPattern RecognitionData AugmentationEye Contact ProcessingEye Contact RecognitionMean Average PrecisionMultimodal Conversation AnalysisMultimodal Machine Learning
In this paper, we present the solution of our team HFUT-VUT for the MultiMediate Grand Challenge 2023 at ACM Multimedia 2023. The solution covers three sub-challenges: bodily behavior recognition, eye contact detection, and next speaker prediction. We select Swin Transformer as the baseline and exploit data augmentation strategies to address the above three tasks. Specifically, we crop the raw video to remove the noise from other parts. At the same time, we utilize data augmentation to improve the generalization of the model. As a result, our solution achieves the best results of 0.6262 for bodily behavior recognition in terms of mean average precision and the accuracy of 0.7771 for eye contact detection on the corresponding test set. In addition, our approach also achieves comparable results of 0.5281 for the next speaker prediction in terms of unweighted average recall.
17
Random Erasing Data Augmentation
Zhun Zhong, Liang Zheng, Guoliang Kang et al. · Proceedings of the AAAI Conference on Artificial Intelligence · 2020 · 2.9K citations · Full text
Ze Liu, Ning Jia, Yue Cao et al. · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · 2022 · 1.9K citations
OpenFace 2.0: Facial Behavior Analysis Toolkit
Tadas Baltrušaitis, Amir Zadeh, Yao Chong Lim et al. · 2018 · 1.8K citations