Reproducibility and responsiveness of evaluative outcome measures. Theoretical considerations illustrated by an empirical example.

TLDR

Background noise in stable patients provides the information needed to quantify reproducibility of evaluative outcome measures. The study proposes using the smallest real difference (SRD) to define responsiveness and outlines basic methods for quantifying reproducibility and responsiveness of evaluative outcome measures. The authors derive the SRD from background noise, construct receiver operating characteristic (ROC) curves for all possible change‑score cutoffs, and illustrate these methods with an empirical example to assess sensitivity and specificity of instruments. The methods enabled evaluation of whether observed change exceeded chance, demonstrated sensitivity and specificity of instruments, and proved useful and complementary in an empirical illustration.

Abstract

This article outlines basic methods for quantifying reproducibility and responsiveness of evaluative outcome measures.The background noise in stable patients provides the desired information to quantify the reproducibility. From this, the smallest real difference (SRD) for longitudinal differences can be derived. We propose to use the SRD to define responsiveness: from all patients who change according to an external criterion, we take the percentage that changes at least SRD on the outcome measure. A more complete picture of the responsiveness of the outcome measure arises when the receiver operating characteristic (ROC) is constructed, corresponding to all possible cut-off change scores. The proposed methods are illustrated with an empirical example.In the illustration the methods appeared to be very useful and complemented each other. We could evaluate whether the observed change score was larger than that expected due to chance. With the methods it was possible to evaluate both the ability of an instrument to detect change if there is a real change in health status (sensitivity to change) and the ability to detect absence of change if there is no real change (specificity to change).We presented the use of SRDs and ROC curves for quantifying reproducibility and responsiveness. We started with the basic notions and arrived at methods that are both understandable and useful.