2024 · 24 citations · 57 references
ComputingComputer ScienceComputer VisionMachine LearningSensorsEmotion RecognitionFace RecognitionFacial ExpressionMotion CaptureMultimodal InteractionTracking SystemsEye Tracking SensorsFacial Expression AssessmentFacial Expression DynamicsFacial Expression RecognitionFacial Motion CapturePower ConsumptionUser Study Evaluation
In this paper, we introduce EyeEcho, a minimally-obtrusive acoustic sensing system designed to enable glasses to continuously monitor facial expressions. It utilizes two pairs of speakers and microphones mounted on glasses, to emit encoded inaudible acoustic signals directed towards the face, capturing subtle skin deformations associated with facial expressions. The reflected signals are processed through a customized machine-learning pipeline to estimate full facial movements. EyeEcho samples at 83.3 Hz with a relatively low power consumption of 167mW. Our user study involving 12 participants demonstrates that, with just four minutes of training data, EyeEcho achieves highly accurate tracking performance across different real-world scenarios, including sitting, walking, and after remounting the devices. Additionally, a semi-in-the-wild study involving 10 participants further validates EyeEcho’s performance in naturalistic scenarios while participants engage in various daily activities. Finally, we showcase EyeEcho’s potential to be deployed on a commercial-off-the-shelf (COTS) smartphone, offering real-time facial expression tracking.
57
Facial Expression Recognition via a Boosted Deep Belief Network
Ping Liu, Shizhong Han, Zibo Meng et al. · 2014 · 668 citations
Look at Boundary: A Boundary-Aware Face Alignment Algorithm
Wenyan Wu, Chen Qian, Shuo Yang et al. · 2018 · 519 citations
Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan et al. · 2016 · 399 citations
Learning Expressionlets on Spatio-temporal Manifold for Dynamic Facial Expression Recognition
Mengyi Liu, Shiguang Shan, Ruiping Wang et al. · 2014 · 396 citations
Real-time expression transfer for facial reenactment
Justus Thies, Michael Zollhöfer, Matthias Nießner et al. · ACM Transactions on Graphics · 2015 · 391 citations