Constructing Better Second Language Assessments Based on Differential Item Functioning Analysis

Abstract

Differential item functioning (DIF) analysis can be used to great advantage in second language (L2) assessments. This study examined the differences in performance on L2 test items between groups from different first language backgrounds and suggested ways of improving L2 assessments. The study examined DIF on L2 (Hebrew) test items for two language groups: Arabic speakers and Russian speakers. Many items (42%) functioned differentially, posing a potential threat to validity. Most problematic were the sentence completion items, which are more loaded on vocabulary and grammar then the other item types. To validate our study results, two special test forms—non-DIF and large DIF—were constructed, based on the results of the study. These test forms were administered during the years 2004–2005. The findings validated the results of the main study. To reduce the number of DIF items in operational tests, we suggest performing DIF analysis on pilot items and changing the ratios of item types included in the test, while preserving the defined test construct.