The aligned rank transform for nonparametric factorial analyses using only anova procedures

TLDR

In HCI, multi‑factor nonparametric data such as error counts, Likert responses, and preference tallies are common, yet standard tests like Friedman cannot assess interactions, and existing methods are scarce and complex. The authors introduce the Aligned Rank Transform (ART) to enable nonparametric factorial analysis in HCI, extending it from two to N factors. ART performs a data‑alignment preprocessing followed by rank averaging, allowing standard ANOVA to be applied, and the authors provide ARTool and ARTweb for implementation. Re‑analysis of published HCI studies demonstrates the ART’s advantages.

Abstract

Nonparametric data from multi-factor experiments arise often in human-computer interaction (HCI). Examples may include error counts, Likert responses, and preference tallies. But because multiple factors are involved, common nonparametric tests (e.g., Friedman) are inadequate, as they are unable to examine interaction effects. While some statistical techniques exist to handle such data, these techniques are not widely available and are complex. To address these concerns, we present the Aligned Rank Transform (ART) for nonparametric factorial data analysis in HCI. The ART relies on a preprocessing step that "aligns" data before applying averaged ranks, after which point common ANOVA procedures can be used, making the ART accessible to anyone familiar with the F-test. Unlike most articles on the ART, which only address two factors, we generalize the ART to N factors. We also provide ARTool and ARTweb, desktop and Web-based programs for aligning and ranking data. Our re-examination of some published HCI results exhibits advantages of the ART.