Is the Simes improved Bonferroni procedure conservative?

Abstract

Simes (1986) proposed a modified Bonferroni procedure for conducting multiple tests of significance. He proved that, when the n test statistics are independent, his procedure has exact size α. Already Hommel(1983) had considered a similar procedure and shown that it need not be conservative. He gave an attainable upper bound for its anticonservativeness. Here we show that, for one-sided tests based on positively correlated bivariate normal test statistics, the procedure is conservative, and for negatively correlated such variables it is anticonservative. Two-sided tests based on bivariate normal variables are always conservative. Some numerical values are given, which show that for all practical purposes the nominal α-value can be used.