Synthetic faces as a lipreading support

Abstract

In the Teleface project the possibility to use a synthetic face as a visual telephone communication aid for hearing impaired persons is evaluated.In an earlier study, NH, a group of normal hearing persons participated.This paper describes the results of two multimodal intelligibility tests with hearing impaired persons, where the additional information provided by a synthetic as well as a natural face is evaluated.In a first round with hearing impaired persons, HI:1, twelve subjects were presented with VCV-syllables and "everyday sentences" together with a questionnaire.The intelligibility score for the VCV-syllables presented as audio alone, was 30%.When adding a synthetic face the score improved to 55% and when instead adding the natural face it was 58%.In a second round, HI:2, fifteen hearing impaired persons were presented with the sentence material and a questionnaire.The audio track was filtered to simulate telephone bandwidth.The intelligibility score for the audio only condition was 57% correctly identified keywords.Together with a synthetic face it was 66% and with a natural face 83%.Answers in the questionnaires were collected and analysed.The general subjective rating of the synthetic face was positive and the subjects would like to use such a type of aid if available.