SUBTLEX-CH: Chinese Word and Character Frequencies Based on Film Subtitles

TLDR

Chinese word frequency data are scarce and lower quality than in other languages, despite the variable’s importance in language research. The authors constructed a subtitle‑based database of 46.8 million characters and 33.5 million words, following prior work in English, French, and Dutch. These frequencies explain more variance in Chinese word naming and lexical decision than written‑text measures, capture contextual diversity and multi‑character usage, and are freely available for research.

Abstract

Background Word frequency is the most important variable in language research. However, despite the growing interest in the Chinese language, there are only a few sources of word frequency measures available to researchers, and the quality is less than what researchers in other languages are used to. Methodology Following recent work by New, Brysbaert, and colleagues in English, French and Dutch, we assembled a database of word and character frequencies based on a corpus of film and television subtitles (46.8 million characters, 33.5 million words). In line with what has been found in the other languages, the new word and character frequencies explain significantly more of the variance in Chinese word naming and lexical decision performance than measures based on written texts. Conclusions Our results confirm that word frequencies based on subtitles are a good estimate of daily language exposure and capture much of the variance in word processing efficiency. In addition, our database is the first to include information about the contextual diversity of the words and to provide good frequency estimates for multi-character words and the different syntactic roles in which the words are used. The word frequencies are freely available for research purposes.