The New Statistics

TLDR

The article argues that research practices must change to address incomplete, untrustworthy literature and the flaws of null‑hypothesis significance testing, advocating a shift toward estimation and other preferred techniques. The paper aims to explain the importance of new statistical practices and provide guidance for their implementation. The authors propose an eight‑step strategy that prespecifies studies, eliminates NHST, and promotes complete reporting and replication to build a cumulative quantitative discipline. The new statistics framework recommends using effect sizes, confidence intervals, and meta‑analysis rather than null‑hypothesis significance testing.

Abstract

We need to make substantial changes to how we conduct research. First, in response to heightened concern that our published research literature is incomplete and untrustworthy, we need new requirements to ensure research integrity. These include prespecification of studies whenever possible, avoidance of selection and other inappropriate data-analytic practices, complete reporting, and encouragement of replication. Second, in response to renewed recognition of the severe flaws of null-hypothesis significance testing (NHST), we need to shift from reliance on NHST to estimation and other preferred techniques. The new statistics refers to recommended practices, including estimation based on effect sizes, confidence intervals, and meta-analysis. The techniques are not new, but adopting them widely would be new for many researchers, as well as highly beneficial. This article explains why the new statistics are important and offers guidance for their use. It describes an eight-step new-statistics strategy for research with integrity, which starts with formulation of research questions in estimation terms, has no place for NHST, and is aimed at building a cumulative quantitative discipline.