A Monte Carlo Study Comparing Various Two-Sample Tests for Differences in Mean

Abstract

A study was conducted on eight tests for differences in means under a variety of simulated experimental situations. Estimates were made of the power of the tests and measures of the extent to which they gave similar results. In particular the performance of a new quick test developed by Neave was studied and was found to be satisfactory: in fact it was by far the best of the quick tests considered. However some of the classical and more general nonparametric tests, such as the runs and the Kolmogorov-Smirnov tests, were found to be less useful when testing for differences in means. Over the range of situations investigated, the Normal Scores test gave the most satisfactory results, followed closely by the Wilcoxon rank-sum test. Even when the populations were normally distributed, these tests were only very slightly inferior to the t-test, and naturally were much superior in the cases of non-normal populations.