The Abuse of Power

TLDR

Statistical power calculations are widely used for experiment planning and post‑experiment interpretation, yet their validity is contested. The authors document the widespread issue and argue that the logic underlying post‑experiment power calculations is flawed. They find that such calculations are fundamentally flawed and provide arguments to support this claim.

Abstract

AbstractIt is well known that statistical power calculations can be valuable in planning an experiment. There is also a large literature advocating that power calculations be made whenever one performs a statistical test of a hypothesis and one obtains a statistically nonsignificant result. Advocates of such post-experiment power calculations claim the calculations should be used to aid in the interpretation of the experimental results. This approach, which appears in various forms, is fundamentally flawed. We document that the problem is extensive and present arguments to demonstrate the flaw in the logic.KEY WORDS: Bioequivalence testingBurden of proofObserved powerRetrospective power analysisStatistical powerType II error