2002 · 33 citations · 11 references
ComputingComputer ScienceComputational LinguisticsNatural Language ProcessingSemanticsDocument ClusteringLanguage ClassificationLanguage ModelingLanguage ModelsText ClusteringTopic ModelingAdaptive InterpolationAdaptive Language LearningClusteringLinear InterpolationSemantic ClusteringSemantic Structure
In this paper we present efficient clustering algorithms for two novel class-based approaches to adaptive language modeling. In contrast to bigram and trigram class models, the proposed classes are related to the distribution and co-occurrence of words within complete text units and are thus mostly of a semantic nature. We introduce adaptation techniques such as the adaptive linear interpolation and an approximation to the minimum discriminant estimation and show how to use the automatically derived semantic structure in order to allow a fast adaptation to some special topic or style. In experiments performed on the Wall-Street-Journal corpus, intuitively convincing semantic classes were obtained. The resulting adaptive language models were significantly better than a standard cache model. Compared to a static model a reduction in perplexity of up to 31% could be achieved.
11
Improved backing-off for M-gram language modeling
Reinhard Kneser, Hermann Ney · 2002 · 1.5K citations