Simulation-Based Power Analysis for Factorial Analysis of Variance Designs

TLDR

ANOVA is widely used, yet performing a priori power analysis for factorial designs is difficult because existing software lacks support for complex within‑participant factors and relies on non‑intuitive effect size metrics. The authors developed the Superpower R package and Shiny apps, along with a tutorial, to enable researchers to conduct simulation‑based power analyses for factorial ANOVA designs with up to three factors. Users input means, standard deviations, and within‑participant correlations, and the simulation computes power for all main effects, interactions, and pairwise comparisons across up to three factors. The tool generates power plots over sample sizes, adjusts for multiple comparisons, and estimates power even when homogeneity or sphericity assumptions are violated.

Abstract

Researchers often rely on analysis of variance (ANOVA) when they report results of experiments. To ensure that a study is adequately powered to yield informative results with an ANOVA, researchers can perform an a priori power analysis. However, power analysis for factorial ANOVA designs is often a challenge. Current software solutions do not allow power analyses for complex designs with several within-participants factors. Moreover, power analyses often need [Formula: see text] or Cohen’s f as input, but these effect sizes are not intuitive and do not generalize to different experimental designs. We have created the R package Superpower and online Shiny apps to enable researchers without extensive programming experience to perform simulation-based power analysis for ANOVA designs of up to three within- or between-participants factors. Predicted effects are entered by specifying means, standard deviations, and, for within-participants factors, the correlations. The simulation provides the statistical power for all ANOVA main effects, interactions, and individual comparisons. The software can plot power across a range of sample sizes, can control for multiple comparisons, and can compute power when the homogeneity or sphericity assumption is violated. This Tutorial demonstrates how to perform a priori power analysis to design informative studies for main effects, interactions, and individual comparisons and highlights important factors that determine the statistical power for factorial ANOVA designs.