Methodological and statistical problems in the construction of composite measurement scales: A survey of six medical and epidemiological journals

TLDR

Composite measurement scales are increasingly used in medicine to assess complex phenomena, yet important measurement properties such as level, content, construct validity, and reliability are often neglected. The study investigates current CMS construction methods by reviewing 46 recent studies and proposes recommendations to guide authors and investigators in selecting and reporting measurement instruments. The authors examined 46 studies published in six major medical and epidemiological journals to assess CMS construction practices. Statistical methods, especially multivariate techniques, are frequently misused, and few studies verify model relevance, assumptions, or perform cross‑validation to prevent overfitting.

Abstract

Composite measurement scales (CMS) are increasingly used in medicine to measure complex phenomena or concepts such as disease risk and severity, physical and psychological functioning and quality of life. To investigate the methodology currently used in the construction of CMS, we examined 46 studies recently published in six major medical and epidemiological journals. Important measurement properties such as measurement level, content and construct validity and reliability are often neglected. Statistical methods, particularly multivariate methods are frequently misused; verifications of model relevance and assumptions, and cross-validations to avoid overfitting are seldom performed. We propose recommendations for the construction and the presentation of CMS, to help authors and investigators to report and choose, respectively, measurement instruments for a complex phenomenon.