A combined modular system for face detection, head pose estimation, face tracking and emotion recognition in thermal infrared images

Abstract

Thermal infrared imaging is an emerging imaging modality allowing capturing heat radiation not detectable in the visible spectrum. In recent years, numerous applications of thermal infrared imaging for the processing of face images have been published. Many of these approaches only allow minimal head movement due to the lack of sufficiently robust face tracking and interpretation algorithms for thermal infrared images. To address this issue, we present a suite of interconnected algorithms for a number of typical facial image processing tasks face detection, head pose estimation, detection and tracking of facial landmarks and facial expression analysis. The modules can be used independently or in combination with each other. For combined use and as a demonstration of the versatility of our solution, we present a multiprocess solution based on a networking middleware that allows using all proposed algorithms to perform real-time face tracking and emotion recognition in thermal infrared images. The full code is made freely available on gitub under GNU license to allow incorporating our solutions into own projects.