Sample size and the probability of a successful trial

TLDR

High statistical power does not guarantee a high probability of trial success; success depends on knowledge of the drug’s ability to deliver the effect used to power the study, building on prior work by O'Hagan et al. The paper distinguishes statistical power from the probability of a successful trial and proposes a framework to compute average success probability while accounting for uncertainty in the treatment effect. The authors present a computational framework, with accompanying code, to estimate average success probability for confirmatory trials. Published in Pharmaceutical Statistics, 2005; 4:187‑201.

Abstract

This paper describes the distinction between the concept of statistical power and the probability of getting a successful trial. While one can choose a very high statistical power to detect a certain treatment effect, the high statistical power does not necessarily translate to a high success probability if the treatment effect to detect is based on the perceived ability of the drug candidate. The crucial factor hinges on our knowledge of the drug's ability to deliver the effect used to power the study. The paper discusses a framework to calculate the 'average success probability' and demonstrates how uncertainty about the treatment effect could affect the average success probability for a confirmatory trial. It complements an earlier work by O'Hagan et al. (Pharmaceutical Statistics 2005; 4:187-201) published in this journal. Computer codes to calculate the average success probability are included.