Connectionist Models of Language Processing

Abstract

Traditional approaches to language processing have been based on explicit, discrete representations which are difficult to learn from a reasonable linguistic environment—hence, it has come to be accepted that much of our linguistic representations and knowledge is innate. With its focus on learning based upon graded, malleable, distributed representations, connectionist modeling has reopened the question of what could be learned from the environment in the absence of detailed innate knowledge. This paper provides an overview of connectionist models of language processing, at both the lexical and sentence levels. Although connectionist models have been applied to the full range of perceptual, cognitive, and motor domains (see McClelland, Rumelhart, & PDP Research Group, 1986; Quinlan, 1991; McLeod, Plunkett, & Rolls, 1998), it is in their application to language that they have evoked the most interest and controversy (e.g., Pinker & Mehler, 1988). This is perhaps not surprising in light of the special role that language plays in human cognition and culture. It also stems in part from the considerable difference in goals and methods between linguistic and psychological approaches to the study of language. This rift goes deeper than a simple dichotomy of emphasizing competence versus performance (Chomsky, 1957)—it cuts to the heart of the question of what it means to know and use a language (Seidenberg, 1997). Traditional approaches to language processing have been based on explicit, discrete representations which are difficult or impossible to learn from a reasonable linguistic environment (Gold, 1967). Therefore, it has come to be accepted that much of our linguistic representations and knowledge is innate. With its focus on learning based upon graded, malleable, distributed representations, connectionist modeling has reopened the question of what could be learned from the environment in the absence of detailed innate knowledge. Although the need to learn internal representations potentially gives connectionist networks great power and flexibility, it also introduces limitations. These limitations are important and, ideally, will reflect limitations observed in human language processing.