Tracking and beamforming for multiple simultaneous speakers with probabilistic data association filters

Abstract

In prior work, we developed a speaker tracking system based on an extended Kalman filter using time delays of arrival (TDOAs) as acoustic features. While this system functioned well, its util-ity was limited to scenarios in which a single speaker was to be tracked. In this work, we remove this restriction by generalizing the IEKF, first to a probabilistic data association filter, which in-corporates a clutter model for rejection of spurious acoustic events, and then to a joint probabilistic data association filter (JPDAF), which maintains a separate state vector for each active speaker. In a set of experiments conducted on seminar and meeting data, the JPDAF speaker tracking system reduced the multiple object track-ing errror from 20.7 % to 14.3 % with respect to the IEKF system. In a set of automatic speech recognition experiments conducted on the output of a 64 channel microphone array which was beam-formed using automatic speaker position estimates, applying the JPDAF tracking system reduced word error rate from 67.3 % to 66.0%. Moreover, the word error rate on the beamformed output was 13.0 % absolute lower than on a single channel of the array. Index Terms: acoustic source localization, Kalman filter, person tracking, far-field speech recognition, microphone arrays