Use of a preliminary test in comparing two sample means

Abstract

Abstract Given independent samples from two distributions it is desired to test equality of means. It is well known that unequal variances can bias the t-test, particularly when the sample sizes differ. Welch's test has been shown to ameliorate this bias. Practitioners often test first for equality of variances, using an alternate test such as Welch's only if equality is rejected. By simulation it is shown that the use of the preliminary test does not remove the bias fully. This is true for normal, uniform, and exponential distributions. Keywords: simulationvariance heterogeneitypoolingBehrens-Fisher problemtwo-sample teststwo-sample t-testWelch's testnon-normal distribution