2004 · 139 citations · 21 references
ComputingComputer ScienceComputational LinguisticsMachine LearningNatural Language ProcessingDeep LearningLanguage ModelingLanguage ModelsPattern RecognitionRepresentation LearningSpeech RecognitionConditional Random FieldsError Rate AnalysisFinite Automata TheoryPerceptronWeighted Finite-state AutomataWord Error Rate
The paper proposes discriminative language modeling for large‑vocabulary speech recognition. It contrasts perceptron and CRF parameter‑estimation methods, encoding models as deterministic weighted finite‑state automata that are intersected with baseline recognizer word lattices. Perceptron training automatically selects a compact feature set in just a few passes, and when used to initialize CRF training yields an extra 0.5 % WER reduction, for a total 1.8 % absolute drop from the 39.2 % baseline.
This paper describes discriminative language modeling for a large vocabulary speech recognition task. We contrast two parameter estimation methods: the perceptron algorithm, and a method based on conditional random fields (CRFs). The models are encoded as deterministic weighted finite state automata, and are applied by intersecting the automata with word-lattices that are the output from a baseline recognizer. The perceptron algorithm has the benefit of automatically selecting a relatively small feature set in just a couple of passes over the training data. However, using the feature set output from the perceptron algorithm (initialized with their weights), CRF training provides an additional 0.5% reduction in word error rate, for a total 1.8% absolute reduction from the baseline of 39.2%.
21
Satish Balay, Shrirang Abhyankar, Mark F. Adams et al. · 2019 · 2K citations
Discriminative training methods for hidden Markov models
Michael Collins · 2002 · 1.9K citations · Full text
Shallow parsing with conditional random fields
Fei Sha, Fernando Pereira · 2003 · 1.3K citations · Full text