An Updated Guideline for Assessing Discriminant Validity

TLDR

Discriminant validity was originally defined through empirical criteria derived from multitrait‑multimethod matrices, but most applied datasets lack MTMM structure, prompting the ad hoc development of numerous assessment techniques. This paper reviews existing definitions and techniques for discriminant validity and offers a generalized definition based on the correlation between two measures after accounting for measurement error. The authors systematically examine proposed techniques, revealing previously unnoticed problems and equivalencies among them. Monte Carlo simulations comparing these methods led to the presentation of two practical techniques, CI CFA (sys) and [Formula] (sys), for assessing discriminant validity.

Abstract

Discriminant validity was originally presented as a set of empirical criteria that can be assessed from multitrait-multimethod (MTMM) matrices. Because datasets used by applied researchers rarely lend themselves to MTMM analysis, the need to assess discriminant validity in empirical research has led to the introduction of numerous techniques, some of which have been introduced in an ad hoc manner and without rigorous methodological support. We review various definitions of and techniques for assessing discriminant validity and provide a generalized definition of discriminant validity based on the correlation between two measures after measurement error has been considered. We then review techniques that have been proposed for discriminant validity assessment, demonstrating some problems and equivalencies of these techniques that have gone unnoticed by prior research. After conducting Monte Carlo simulations that compare the techniques, we present techniques called CI CFA (sys) and [Formula: see text](sys) that applied researchers can use to assess discriminant validity.