Speech segregation based on sound localization

TLDR

Speech segregation at a cocktail party requires selective attention to a single voice, but simulating this perceptual ability remains a great challenge. The study proposes a supervised learning method that separates target speech from interference using ITD and IID cues. The method models an ideal time–frequency binary mask based on auditory masking, then uses pattern classification to estimate these masks from ITD and IID features. Evaluation shows the system generates masks close to ideal, significantly outperforms prior methods, and yields large intelligibility gains for listeners.

Abstract

At a cocktail party, one can selectively attend to a single voice and filter out all the other acoustical interferences. How to simulate this perceptual ability remains a great challenge. This paper describes a novel, supervised learning approach to speech segregation, in which a target speech signal is separated from interfering sounds using spatial localization cues: interaural time differences (ITD) and interaural intensity differences (IID). Motivated by the auditory masking effect, the notion of an “ideal” time–frequency binary mask is suggested, which selects the target if it is stronger than the interference in a local time–frequency (T–F) unit. It is observed that within a narrow frequency band, modifications to the relative strength of the target source with respect to the interference trigger systematic changes for estimated ITD and IID. For a given spatial configuration, this interaction produces characteristic clustering in the binaural feature space. Consequently, pattern classification is performed in order to estimate ideal binary masks. A systematic evaluation in terms of signal-to-noise ratio as well as automatic speech recognition performance shows that the resulting system produces masks very close to ideal binary ones. A quantitative comparison shows that the model yields significant improvement in performance over an existing approach. Furthermore, under certain conditions the model produces large speech intelligibility improvements with normal listeners.