Variable n-grams and extensions for conversational speech language modeling

Abstract

Recent progress in variable n-gram language modeling provides an efficient representation of n-gram models and makes training of higher order n grams possible. We apply the variable n-gram design algorithm to conversational speech, extending the algorithm to learn skips and context-dependent classes to handle conversational speech characteristics such as filler words, repetitions, and other disfluencies. Experiments show that using the extended variable n-gram results in a language model that captures 4-gram context with less than half the parameters of a standard trigram while also improving the test perplexity and recognition accuracy.