An Application of Measurement Methods to Improve the Quantitative Nature of Pilot Rating Scales

Abstract

Although various forms of pilot rating scales have been widely used, they have at the same time been criticized increasingly for supposed deficiencies. Thus, contemporary scale problems include wording ambiguity, the dual mission character of some scales, and a lack of information about the quantitative character of the scale continuum. In an attempt to quantize rating scales, a semantic experiment was conducted that allowed psychological measurement techniques to be applied to scale the wording used in rating scales. The results of the application of the method of successive intervals indicate that contemporary scale data can be averaged directly with little error if a reliable estimate of the mean is available. However, the number of samples necessary to obtain a reliable estimate depends on the rating itself, and increases as the rating becomes worse, so that the design of an experiment would need to depend on the outcome of the same experiment. The problem could be avoided by constructing a scale based on the successive interval scale values, where variability along the scale is constant.