Comparisons of Treatments After an Analysis of Variance in Ecology

TLDR

Planned comparisons in ecology offer advantages, and the philosophy of hypothesis testing with unplanned multiple comparisons is examined in relation to confidence intervals and statistical estimation. The study reviews the statistical literature on tests to compare treatments after analysis of variance and examines their use in ecology. The authors recommend specific post‑ANOVA tests for unplanned multiple comparisons—covering parametric and nonparametric cases, equal and unequal sample sizes, homogeneous and heterogeneous variances, and non‑independent means—provide formulae and a worked example, and investigate assumption violations via simulations to suggest strategies. Monte Carlo simulations reveal that many commonly used post‑ANOVA tests are inappropriate or inefficient, and that violations of assumptions, especially variance heterogeneity, require particular strategies.

Abstract

The statistical literature on tests to compare treatments after the analysis of variance is reviewed, and the use of these tests in ecology is examined. Monte Carlo simulations on normal and lognormal data indicate that many of the tests commonly used are inappropriate or inefficient. Particular tests are recommended for unplanned multiple comparisons on the basis of controlling experimentwise type I error rate and providing maximum power. These include tests for parametric and nonparametric cases, equal and unequal sample sizes, homogeneous and heterogeneous variances, non—independent means (repeated measures or adjusted means), and comparing treatments to a control. Formulae and a worked example are provided. The problem of violations of assumptions, especially variance heterogeneity, was investigated using simulations, and particular strategies are recommended. The advantages and use of planned comparisons in ecology are discussed, and the philosophy of hypothesis testing with unplanned multiple comparisons is considered in relation to confidence intervals and statistical estimation.