Automatically Assessing Lexical Sophistication: Indices, Tools, Findings, and Application

TLDR

The study investigates lexical sophistication as a measure of second‑language lexical and speaking proficiency. The authors introduce TAALES, a freely available, cross‑platform tool that automatically computes 135 lexical indices and outputs results to CSV for batch processing. The indices explained 47.5 % of lexical proficiency variance and 48.7 % of speaking proficiency variance, underscoring their value for L2 assessment. The paper discusses limitations and future research directions.

Abstract

This study explores the construct of lexical sophistication and its applications for measuring second language lexical and speaking proficiency. In doing so, the study introduces the Tool for the Automatic Analysis of LE xical Sophistication ( TAALES ), which calculates text scores for 135 classic and newly developed lexical indices related to word frequency, range, bigram and trigram frequency, academic language, and psycholinguistic word information. TAALES is freely available; runs on Windows, Mac, and Linux operating systems; and has a simple graphic user interface that allows for batch processing of .txt files. The tool is fast, reliable, and outputs results to a comma‐separated value file that can be accessed using spreadsheet software. The study examines the ability of TAALES indices to explain the variance in human judgments of lexical proficiency and speaking proficiency for second language (L2) learners. Overall, these indices were able to explain 47.5% of the variance in holistic scores of lexical proficiency and 48.7% of the variance in holistic scores of speaking proficiency. This study has important implications for second language acquisition, for assessing L2 learners' productive skills (writing and speaking), and for L2 pedagogy. Limitations and future directions are also discussed.