Examining Multiple Comparison Procedures According to Error Rate, Power Type and False Discovery Rate

Abstract

Examining pairwise differences between means is a common practice of applied researchers, and the selection of an appropriate multiple comparison procedure (MCP) is important for analyzing pairwise comparisons. This study examines the performance of MCPs under the assumption of homogeneity of variances for various numbers of groups with equal and unequal sample sizes via a simulation study. MCPs are compared according to type I error rate, power type and false discovery rate (FDR). Results show that the LSD and Duncan procedures have high error rates and Scheffe’s procedure has low power; no remarkable differences between the other procedures considered were identified.