An Evaluation of Five Linear Equating Methods for the NEAT Design

Abstract

This study uses the results of two previous papers (Kane, Mroch, Suh, & Ripkey, this issue; Suh, Mroch, Kane, & Ripkey, this issue) and the literature on linear equating to evaluate five linear equating methods along several dimensions, including the plausibility of their assumptions and their levels of bias and root mean squared difference (RMSD). The methods all employ non-equivalent groups anchor test (NEAT) design, but make different assumptions about the empirical relationship to be generalized across groups. The analyses indicate that the assumptions employed in Levine Observed-score and Levine True-score methods are more plausible than those for a Tucker, Tucker-like, and Chained Linear method, and that the Levine methods generally have lower levels of bias and RMSD than the other three methods. Furthermore, the methods that employed a chained linear relationship (CLR) approach, in which observed relationships between total test scores and anchor test scores are generalized across groups taking various tests, are found to be more consistent with programs in which a series of test forms administered over a period of years are equated to each other, than a parameter substitution (PS) approach, which estimates results for specific synthetic populations. It is argued that the Levine Observed-score and Levine True-score methods have strong advantages over the other methods studied, unless the groups taking the tests to be equated are known to be very similar.