Overlap-aware diarization: resegmentation using neural end-to-end\n overlapped speech detection

Abstract

We address the problem of effectively handling overlapping speech in a\ndiarization system. First, we detail a neural Long Short-Term Memory-based\narchitecture for overlap detection. Secondly, detected overlap regions are\nexploited in conjunction with a frame-level speaker posterior matrix to make\ntwo-speaker assignments for overlapped frames in the resegmentation step. The\noverlap detection module achieves state-of-the-art performance on the AMI,\nDIHARD, and ETAPE corpora. We apply overlap-aware resegmentation on AMI,\nresulting in a 20% relative DER reduction over the baseline system. While this\napproach is by no means an end-all solution to overlap-aware diarization, it\nreveals promising directions for handling overlap.\n