An investigation of acoustic models for multilingual code-switching

Abstract

Abstract Multilingual speech processing continues to develop as speechtechnology spreads to heterogeneous clients and applications.We address a distinct problem of code-switching — the spon-taneous but occasional use, within speech in one language (re-ferred to as L 1 ), of words, phrases, expressions or idioms froma second language (L 2 ). We examine two alternatives for mod-eling the acoustics of such words: creation of L 1 pronunciationsfor the out-of-language (OOL) words for use with L 1 acousticmodels, and retention of their L 2 pronunciations for use withmultilingual acoustic models. We test the hypothesis that thelatter is a better acoustic model for OOL words. We develop aset of lexica in IPA form, a global phoneme inventory, and han-dle the problem of L 2 word pronunciation by creating linguis-tically motivated pairwise mappings. We show that retentionof L 2 pronunciations with multilingual acoustic models betterexplains the observations when restricted to a forced alignment.Index Terms: Cross-lingual and multi-lingual processing, Au-tomatic speech recognition, Accent and language identification,Spoken language resources and annotation