PLS, Small Sample Size, and Statistical Power in MIS Research

TLDR

MIS scholars often regard PLS as superior to regression or LISREL for small‑sample analyses. The study sought to compare path estimates and statistical power across PLS, regression, and LISREL for different sample sizes and effect sizes. Monte Carlo simulations were employed to evaluate the three techniques at sample sizes of 40, 90, 150, and 200 and effect sizes ranging from large to none. The results indicate that PLS with bootstrapping offers no special power advantage at small sample sizes, and none of the methods reliably detect small or medium effects in simple normally distributed models, contradicting prevailing MIS claims.

Abstract

There is a pervasive belief in the Management Information Systems (MIS) field that Partial Least Squares (PLS) has special abilities that make it more appropriate than other techniques, such as multiple regression and LISREL, when analyzing small sample sizes. We conducted a study using Monte Carlo simulation to compare these three relatively popular techniques for modeling relationships among variables under varying sample sizes (N = 40, 90, 150, and 200) and varying effect sizes (large, medium, small and no effect). The focus of the analysis was on comparing the path estimates and the statistical power for each combination of technique, sample size, and effect size. The results suggest that PLS with bootstrapping does not have special abilities with respect to statistical power at small sample sizes. In fact, for simple models with normally distributed data and relatively reliable measures, none of the three techniques have adequate power to detect small or medium effects at small sample sizes. These findings run counter to extant suggestions in MIS literature.