Optimizing rating scale category effectiveness.

TLDR

Rating scales extract more information than dichotomous responses, and the guidelines discussed apply broadly across analytical methods beyond Rasch analysis. The study asks whether this richer information improves measurement accuracy and precision, and proposes eight guidelines to optimize rating‑scale category cooperation. The guidelines rely on category frequency, ordering, rating‑to‑measure coherence, and scale quality, encourage recategorization or reconceptualization, and are demonstrated on two published datasets.

Abstract

Rating scales are employed as a means of extracting more information out of an item than would be obtained from a mere "yes/no", "right/wrong" or other dichotomy. But does this additional information increase measurement accuracy and precision? Eight guidelines are suggested to aid the analyst in optimizing the manner in which rating scales categories cooperate in order to improve the utility of the resultant measures. Though these guidelines are presented within the context of Rasch analysis, they reflect aspects of rating scale functioning which impact all methods of analysis. The guidelines feature rating-scale-based data such as category frequency, ordering, rating-to-measure inferential coherence, and the quality of the scale from measurement and statistical perspectives. The manner in which the guidelines prompt recategorization or reconceptualization of the rating scale is indicated. Utilization of the guidelines is illustrated through their application to two published data sets.