Head-Driven Statistical Models for Natural Language Parsing

TLDR

The article introduces three statistical models for natural language parsing and aims to understand and compare their performance on the Penn Wall Street Journal Treebank. The models extend probabilistic context‑free grammars to lexicalized, head‑centered, top‑down derivations, encoding X‑bar structure, subcategorization, adjunct placement, lexical dependencies, and wh‑movement as probabilities conditioned on lexical heads, and are evaluated through parsing accuracy experiments and linguistic analyses. Evaluation on the Penn Wall Street Journal Treebank shows that the models achieve accuracy competitive with existing parsing models.

Abstract

This article describes three statistical models for natural language parsing. The models extend methods from probabilistic context-free grammars to lexicalized grammars, leading to approaches in which a parse tree is represented as the sequence of decisions corresponding to a head-centered, top-down derivation of the tree. Independence assumptions then lead to parameters that encode the X-bar schema, subcategorization, ordering of complements, placement of adjuncts, bigram lexical dependencies, wh-movement, and preferences for close attachment. All of these preferences are expressed by probabilities conditioned on lexical heads. The models are evaluated on the Penn Wall Street Journal Treebank, showing that their accuracy is competitive with other models in the literature. To gain a better understanding of the models, we also give results on different constituent types, as well as a breakdown of precision/recall results in recovering various types of dependencies. We analyze various characteristics of the models through experiments on parsing accuracy, by collecting frequencies of various structures in the treebank, and through linguistically motivated examples. Finally, we compare the models to others that have been applied to parsing the treebank, aiming to give some explanation of the difference in performance of the various models.