A Rasch-based validation of the Vocabulary Size Test

TLDR

The Vocabulary Size Test is designed to measure written receptive knowledge of the first 14,000 words of English. The primary purpose of this study was to provide preliminary validity evidence for a 140‑item form of the Vocabulary Size Test. The study involved 19 native English speakers and 178 native Japanese speakers and applied Rasch‑model analyses focusing on Messick’s validation framework. The Rasch analyses showed good item fit, high unidimensionality (85.6 % variance explained), strong measurement invariance (Pearson correlations 0.91–0.96), and reliability >0.96, indicating that the Vocabulary Size Test is a precise, valid instrument that extends the measurement range of written receptive vocabulary.

Abstract

The primary purpose of this study was to provide preliminary validity evidence for a 140-item form of the Vocabulary Size Test, which is designed to measure written receptive knowledge of the first 14,000 words of English. Nineteen native speakers of English and 178 native speakers of Japanese participated in the study. Analyses based on the Rasch model were focused on several aspects of Messick’s validation framework. The findings indicated that (1) the items and examinees generally performed as predicted by a priori hypotheses, (2) the overwhelming majority of the items displayed good fit to the Rasch model, (3) the items displayed a high degree of unidimensionality with the Rasch model accounting for 85.6% of the variance, (4) the items showed a strong degree of measurement invariance with disattenuated Pearson correlations for person measures estimated with different sets of items of 0.91 and 0.96, and (5) various combinations of items provided precise measurement for this sample of examinees as indicated by Rasch reliability indices >0.96. The Vocabulary Size Test provides teachers and researchers with a new instrument that greatly extends the range of measurement provided by other measures of written receptive vocabulary size.