Power and sensitivity of alternative fit indices in tests of measurement invariance.

TLDR

Confirmatory factor analytic tests of measurement invariance based on the chi‑square statistic are highly sensitive to sample size. The authors investigated the performance of alternative fit indices with simulated data known to lack invariance. The study employed simulated non‑invariant data to compare alternative fit indices against chi‑square tests of measurement invariance. The study found that alternative fit indices are far less sensitive to sample size and more sensitive to lack of invariance than chi‑square tests, and that reporting changes in CFI and McDonald’s NCI—especially condition‑specific NCI changes—offers superior detection of measurement invariance violations.

Abstract

Confirmatory factor analytic tests of measurement invariance (MI) based on the chi-square statistic are known to be highly sensitive to sample size. For this reason, G. W. Cheung and R. B. Rensvold (2002) recommended using alternative fit indices (AFIs) in MI investigations. In this article, the authors investigated the performance of AFIs with simulated data known to not be invariant. The results indicate that AFIs are much less sensitive to sample size and are more sensitive to a lack of invariance than chi-square-based tests of MI. The authors suggest reporting differences in comparative fit index (CFI) and R. P. McDonald's (1989) noncentrality index (NCI) to evaluate whether MI exists. Although a general value of change in CFI (.002) seemed to perform well in the analyses, condition specific change in McDonald's NCI values exhibited better performance than a single change in McDonald's NCI value. Tables of these values are provided as are recommendations for best practices in MI testing.