Efficient Graph-Based Semi-Supervised Learning of Structured Tagging Models

TLDR

We present a scalable semi‑supervised algorithm for training conditional random fields (CRF) to perform part‑of‑speech tagging. The method constructs a similarity graph over n‑grams and uses it during training to smooth CRF state posteriors on the target domain, while inference remains standard. Experiments show the approach scales to very large problems and achieves significantly higher accuracy on the target domain.

Abstract

We describe a new scalable algorithm for semi-supervised training of conditional random fields (CRF) and its application to part-of-speech (POS) tagging. The algorithm uses a similarity graph to encourage similar n-grams to have similar POS tags. We demonstrate the efficacy of our approach on a domain adaptation task, where we assume that we have access to large amounts of unlabeled data from the target domain, but no additional labeled data. The similarity graph is used during training to smooth the state posteriors on the target domain. Standard inference can be used at test time. Our approach is able to scale to very large problems and yields significantly improved target domain accuracy.