Methodology Review: Assessing Unidimensionality of Tests and ltenls

TLDR

Reliability is defined as the ratio of true‑score variance to observed variance, while internal consistency and homogeneity refer to item‑correlation patterns; unidimensionality is defined as a single latent trait underlying the data. The paper reviews methods for assessing unidimensionality and clarifies the term’s definition relative to related concepts such as reliability, internal consistency, and homogeneity. The authors review indices based on answer patterns, reliability, components, factor analysis, and latent traits to assess unidimensionality. The study finds that many existing indices lack a solid rationale, and that those based on residual size after fitting a two‑ or three‑parameter latent trait model are most useful for detecting unidimensionality.

Abstract

Various methods for determining unidimensionality are reviewed and the rationale of these methods is as sessed. Indices based on answer patterns, reliability, components and factor analysis, and latent traits are reviewed. It is shown that many of the indices lack a rationale, and that many are adjustments of a previous index to take into account some criticisms of it. After reviewing many indices, it is suggested that those based on the size of residuals after fitting a two- or three-parameter latent trait model may be the most useful to detect unidimensionality. An attempt is made to clarify the term unidimensional, and it is shown how it differs from other terms often used inter changeably such as reliability, internal consistency, and homogeneity. Reliability is defined as the ratio of true score variance to observed score variance. Inter nal consistency denotes a group of methods that are intended to estimate reliability, are based on the vari ances and the covariances of test items, and depend on only one administration of a test. Homogeneity seems to refer more specifically to the similarity of the item correlations, but the term is often used as a synonym for unidimensionality. The usefulness of the terms in ternal consistency and homogeneity is questioned. Uni dimensionality is defined as the existence of one latent trait underlying the data.