2010 · 299 citations · 25 references
ComputingComputer ScienceComputational LinguisticsMachine LearningNatural Language ProcessingDeep LearningPattern RecognitionRepresentation LearningSequence LabelingSequence Pattern RecognitionSupervised LearningConditional Random FieldsLabelsRegularization
Conditional Random Fields are popular for supervised sequence labeling because they can handle large label spaces and capture structural dependencies, yet their parameter count and computational cost grow quadratically with the number of labels even in simple linear‑chain models. This study aims to train very large CRFs with hundreds of output labels and billions of features. Efficiency is achieved by exploiting sparsity from an ℓ‑penalty regularization, and the authors compare three recent implementations of this strategy. Experiments show that such large CRFs can be trained efficiently, achieve higher accuracy, and produce compact parameter sets.
Conditional Random Fields (CRFs) are a widely-used approach for supervised sequence labelling, notably due to their ability to handle large description spaces and to integrate structural dependency between labels. Even for the simple linear-chain model, taking structure into account implies a number of parameters and a computational effort that grows quadratically with the cardinality of the label set. In this paper, we address the issue of training very large CRFs, containing up to hundreds output labels and several billion features. Efficiency stems here from the sparsity induced by the use of a l penalty term. Based on our own implementation, we compare three recent proposals for implementing this regularization strategy. Our experiments demonstrate that very large CRFs can be trained efficiently and that very large models are able to improve the accuracy, while delivering compact parameter sets.
25