The arcsine is asinine: the analysis of proportions in ecology

TLDR

The arcsine square‑root transformation has long been the standard for analyzing proportional data in ecology, yet logistic regression offers greater interpretability and power for binomial data and avoids nonsensical predictions and overdispersion issues that can arise with the arcsine transform in non‑binomial contexts. We argue that the arcsine transform should not be used for proportional data and propose the logit transformation as a preferable alternative. The authors illustrate these advantages with examples comparing untransformed, arcsine‑ and logit‑transformed linear models, logistic regression, and logistic regression with random effects. Simulations demonstrate that logistic regression typically yields greater statistical power than the other methods examined.

Abstract

The arcsine square root transformation has long been standard procedure when analyzing proportional data in ecology, with applications in data sets containing binomial and non-binomial response variables. Here, we argue that the arcsine transform should not be used in either circumstance. For binomial data, logistic regression has greater interpretability and higher power than analyses of transformed data. However, it is important to check the data for additional unexplained variation, i.e., overdispersion, and to account for it via the inclusion of random effects in the model if found. For non-binomial data, the arcsine transform is undesirable on the grounds of interpretability, and because it can produce nonsensical predictions. The logit transformation is proposed as an alternative approach to address these issues. Examples are presented in both cases to illustrate these advantages, comparing various methods of analyzing proportions including untransformed, arcsine- and logit-transformed linear models and logistic regression (with or without random effects). Simulations demonstrate that logistic regression usually provides a gain in power over other methods.