2022 · 10 citations · 26 references
ComputingComputer ScienceArtificial IntelligenceComputer VisionMachine LearningDeep LearningEmotion RecognitionFace RecognitionFacial ExpressionMultimodal InteractionPattern RecognitionAudio-visual QualityFacial Affect RecognitionFacial Expression InputFacial Expression PerceptionFacial Expression ProcessingFacial Expression RepresentationFully Convolutional NetworksSubjective Quality Assessment
Online videoconferencing applications have become more popular and occupy a significant portion of consumer Internet traffic. Thus, it is crucial to assess the Quality of Experience (QoE) for these applications to effectively implementing high-quality services. Although the major approach to assess the QoE is the conduction of a subjective quality assessment, this method relies on user’s feedback and it is not suitable for real-time management systems. For this reason, in this paper, we propose alternative QoE estimation models based on facial expression features extracted from the face of the users during audiovisual conversations with a WebRTC-based telemeeting service. We first conducted a subjective quality assessment to investigate the QoE of two-party audiovisual conversations impaired by poor network conditions. Then, we used the extracted facial expression features obtained by observing the test participants to train Machine Learning (ML) algorithms and a Fully Convolutional Network (FCN) with the aim to estimate the perceived QoE solely on the base of these features. Finally, we discuss and compare the QoE estimation performance of the proposed models in terms of accuracy, precision, recall, and F1-score metrics. In particular, the ML model (Support Vector Machine) and the FCN achieved the mean estimation accuracy of 0.78 and 0.70, respectively.
26
Fully convolutional networks for semantic segmentation
Jonathan Long, Evan Shelhamer, Trevor Darrell · 2015 · 36.9K citations
ADASYN: Adaptive synthetic sampling approach for imbalanced learning
Haibo He, Yang Bai, Edwardo A. Garcia et al. · 2008 · 4.4K citations
OpenFace 2.0: Facial Behavior Analysis Toolkit
Tadas Baltrušaitis, Amir Zadeh, Yao Chong Lim et al. · 2018 · 1.8K citations
Rendering of Eyes for Eye-Shape Registration and Gaze Estimation
Erroll Wood, Tadas Baltruaitis, Xucong Zhang et al. · 2015 · 317 citations