OpenFace 2.0: Facial Behavior Analysis Toolkit

TLDR

Automatic facial behavior analysis has gained increasing interest in recent years. OpenFace 2.0 is introduced as a toolkit for computer vision, machine learning, and affective computing researchers and developers to build interactive applications based on facial behavior analysis. OpenFace 2.0 extends the original toolkit with improved facial landmark detection, head‑pose estimation, facial action‑unit recognition, and eye‑gaze estimation. Its core algorithms achieve state‑of‑the‑art performance across landmark detection, pose, action units, and gaze, run in real time from a webcam, and are freely available for research.

Abstract

Over the past few years, there has been an increased interest in automatic facial behavior analysis and understanding. We present OpenFace 2.0 - a tool intended for computer vision and machine learning researchers, affective computing community and people interested in building interactive applications based on facial behavior analysis. OpenFace 2.0 is an extension of OpenFace toolkit and is capable of more accurate facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation. The computer vision algorithms which represent the core of OpenFace 2.0 demonstrate state-of-the-art results in all of the above mentioned tasks. Furthermore, our tool is capable of real-time performance and is able to run from a simple webcam without any specialist hardware. Finally, unlike a lot of modern approaches or toolkits, OpenFace 2.0 source code for training models and running them is freely available for research purposes.