2003 · 469 citations · 9 references
ComputingComputer ScienceComputational LinguisticsMachine LearningNatural Language ProcessingHidden Markov ModelsKernel MethodsNamed Entity RecognitionPart-of-speech TaggingPattern RecognitionSupport Vector MachinesBoostingConditional Random FieldsMaximum Margin ClassificationSupport Vector Machine Classification
HM‑SVMs offer advantages over prior methods such as CRFs, MEMMs, and label‑sequence boosting, including the ability to handle overlapping features. The paper introduces Hidden Markov Support Vector Machines, a novel discriminative learning method combining SVMs and HMMs for label sequences. The method models label dependencies with Viterbi decoding and learns discriminative parameters via a maximum‑soft‑margin criterion. Experiments on named entity recognition and part‑of‑speech tagging show that HM‑SVMs can learn non‑linear discriminant functions with kernels and achieve competitive performance.
This paper presents a novel discriminative learning technique for label sequences based on a combination of the two most successful learning algorithms, Support Vector Machines and Hidden Markov Models which we call Hidden Markov Support Vector Machine. The proposed architecture handles dependencies between neighboring labels using Viterbi decoding. In contrast to standard HMM training, the learning procedure is discriminative and is based on a maximum/soft margin criterion. Compared to previous methods like Conditional Random Fields, Maximum Entropy Markov Models and label sequence boosting, HM-SVMs have a number of advantages. Most notably, it is possible to learn non-linear discriminant functions via kernel functions. At the same time, HM-SVMs share the key advantages with other discriminative methods, in particular the capability to deal with overlapping features. We report experimental evaluations on two tasks, named entity recognition and part-of-speech tagging, that demonstrate the competitiveness of the proposed approach.
9
Training support vector machines: an application to face detection
E. Osuna, Robert M. Freund · 2002 · 2.5K citations
Discriminative training methods for hidden Markov models
Michael Collins · 2002 · 1.9K citations · Full text
The Use of Classifiers in Sequential Inference
Vasin Punyakanok, Dan Roth · ArXiv.org · 2001 · 171 citations · Full text