Pseudoreplication and the Design of Ecological Field Experiments

TLDR

Pseudoreplication occurs when inferential statistics are applied to experiments lacking independent treatment replicates, leading to inappropriate error terms and vulnerability to chance events that can bias results. The study argues that interspersion of treatments is essential to guard against pseudoreplication and preexisting gradients, and proposes recommendations for statisticians and journal editors to enhance ecologists’ design and statistical literacy. The authors review key aspects of controlled experiments, noting that in small studies interspersion may require abandoning strict randomization, and explain the trade‑off between interspersion and randomization by differentiating pre‑layout and layout‑specific type I error rates. Analysis of 176 ecological studies found pseudoreplication in 27% overall, rising to 48% among those using inferential statistics, with particularly high rates in marine benthos and small mammal research.

Abstract

Pseudoreplication is defined as the use of inferential statistics to test for treatment effects with data from experiments where either treatments are not replicated (though samples may be) or replicates are not statistically independent. In ANOVA terminology, it is the testing for treatment effects with an error term inappropriate to the hypothesis being considered. Scrutiny of 176 experimental studies published between 1960 and the present revealed that pseudoreplication occurred in 27% of them, or 48% of all such studies that applied inferential statistics. The incidence of pseudoreplication is especially high in studies of marine benthos and small mammals. The critical features of controlled experimentation are reviewed. Nondemonic intrusion is defined as the impingement of chance events on an experiment in progress. As a safeguard against both it and preexisting gradients, interspersion of treatments is argued to be an obligatory feature of good design. Especially in small experiments, adequate interspersion can sometimes be assured only by dispensing with strict randomization procedures. Comprehension of this conflict between interspersion and randomization is aided by distinguishing pre—layout (or conventional) and layout—specific alpha (probability of type I error). Suggestions are offered to statisticians and editors of ecological journals as to how ecologists' understanding of experimental design and statistics might be improved.