Equidistance of Likert-Type Scales and Validation of Inferential Methods Using Experiments and Simulations

Abstract

Abstract: Likert-type data are often assumed to be equidistant by applied researchers so that they can use parametric\nmethods to analyse the data. Since the equidistance assumption rarely is tested, the validity of parametric analyses of\nLikert-type data is often unclear. This paper consists of two parts where we deal with this validity problem in two different\nrespects. In the first part, we use an experimental design to show that the perceived distance between scale points on a\nregular five-point Likert-type scale depends on how the verbal anchors are used. Anchors only at the end points create a\nrelatively larger perceived distance between points near the ends of the scale than in the middle (end-of-scale effect),\nwhile anchors at all points create a larger perceived distance between points in the middle of the scale (middle-of-scale\neffect). Hence, Likert-type scales are generally not perceived as equidistant by subjects. In the second part of the paper, we\nuse Monte Carlo simulations to explore how parametric methods commonly used to compare means between several\ngroups perform in terms of actual significance and power when data are assumed to be equidistant even though they are\nnot. The results show that the preferred statistical method to analyse Likert-type data depends on the nature of their nonequidistance\nas well as their skewness. Under middle-of-scale effect, the omnibus one-way ANOVA works best when data\nare relatively symmetric. However, the Kruskal-Wallis test works better when data are skewed except when sample sizes\nare unequal, in which case the Brown-Forsythe test is better. Under end-of-scale effect, on the other hand, the Kruskal-\nWallis test should be preferred in most cases when data are at most moderately skewed. When data are heavily skewed,\nANOVA works best unless when sample sizes are unequal, in which case the Brown-Forsythe test should be preferred.