A facial expression recognition for a speaker of a phoneme of vowel using thermal image processing and a speech recognition system

Abstract

We have proposed a method for facial expression recognition for a speaker using thermal image processing and a speech recognition system. In this study, using a speech recognition system, we have improved our system to save thermal images at the three timing positions of just before speaking, and just speaking phonemes of the first and last vowels. In this method, the facial expressions were discriminable with the average recognition accuracy of 81% for four subjects, when he or she exhibited one of the intentional facial expressions of ldquoangryrdquo, ldquohappyrdquo, ldquoneutralrdquo, ldquosadrdquo, and ldquosurpriserdquo.