Proceedings of the National Academy of Sciences · 2020 · 75 citations · 42 references
Language StudiesComputer ScienceLinguisticsComputational LinguisticsNatural Language ProcessingSyntaxGrammarLanguage ModelingLanguage UniversalsLinguistic UniversalsSyntactic ComplexityWord OrderCross-linguistic Word Order UniversalsFormal Explanations Of UniversalsGrammar OptimizationOrder ParametersOrder SensitivitySyntactic StructuresUniversals Of Morphological ComplexityWord Order Complexity
The universal properties of human languages have been the subject of intense study across the language sciences. We report computational and corpus evidence for the hypothesis that a prominent subset of these universal properties-those related to word order-result from a process of optimization for efficient communication among humans, trading off the need to reduce complexity with the need to reduce ambiguity. We formalize these two pressures with information-theoretic and neural-network models of complexity and ambiguity and simulate grammars with optimized word-order parameters on large-scale data from 51 languages. Evolution of grammars toward efficiency results in word-order patterns that predict a large subset of the major word-order correlations across languages.
42
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
A Mathematical Theory of Communication
Claude E. Shannon · Bell System Technical Journal · 1948 · 80.7K citations
Three Factors in Language Design
Noam Chomsky · Linguistic Inquiry · 2005 · 2.6K citations
Natural language and natural selection
Steven Pinker, Paul Bloom · Behavioral and Brain Sciences · 1990 · 2.5K citations