Hand and Lip Desynchronization Analysis in French Cued Speech: Automatic Temporal Segmentation of Hand Flow

Abstract

In the context of cued speech gesture phonetic translation, the automatic recognition of lip and hand movements is a key factor. The hand and the lip parameters are not synchronized, thus the fusion of the two channels (hand and lips) needs the knowledge of the desynchronized delay. This contribution focuses on the presentation of an automatic algorithm for temporal segmentation of the hand cue information based on Gaussian modeling of the hand position and minimum of velocity. The segmentation delivers the beginning of the hand transition and the instant of attained position. The hand segmentation is used to calculate the delay between hand and lip targets, in relation with the corresponding acoustic realization in the case of French CV syllables extracted from a corpus of phrases uttered and coded by a cued speech speaker. This study confirms in a more complex context the importance of the instant of attained hand position as pointed out by Attina and colleagues, in terms of control and for the fusion process