A Multiple Testing Procedure for Clinical Trials

TLDR

The authors introduce a multiple testing procedure for comparing two treatments with dichotomous, immediate outcomes. The method uses a Pearson chi‑square statistic computed on all accumulated data, with a pre‑specified maximum number of tests N and fixed sample size m1+m2 per test. The procedure maintains type‑I error and power comparable to a single chi‑square test on N(m1+m2) observations, while permitting early stopping to mitigate ethical concerns.

Abstract

A multiple testing procedure is proposed for comparing two treatments when response to treatment is both dichotomous (i.e., success or failure) and immediate. The proposed test statistic for each test is the usual (Pearson) chi-square statistic based on all data collected to that point. The maximum number (N) of tests and the number (m1 + m2) of observations collected between successive tests is fixed in advance. The overall size of the procedure is shown to be controlled with virtually the same accuracy as the single sample chi-square test based on N(m1 + m2) observations. The power is also found to be virtually the same. However, by affording the opportunity to terminate early when one treatment performs markedly better than the other, the multiple testing procedure may eliminate the ethical dilemmas that often accompany clinical trials.