Power and sample size for nested analysis of molecular variance

Abstract

Analysis of molecular variance (AMOVA) is a widely used tool for quantifying the contribution of various levels of population structure to patterns of genetic variation. Implementations of AMOVA use permutation tests to evaluate null hypotheses of no population structure within groups and between groups. With few populations per group, between-group structure might be impossible to detect because only a few permutations of the sampled populations are possible. In fact, with fewer than six total populations, permutation tests will never result in P-values <0.05 for higher-level population structure. I present minimum numbers of replicates calculated from multinomial coefficients and an r script that can be used to evaluate the minimum P-value for any sampling scheme. While it might seem counterintuitive that a large sample of individuals is uninformative about hierarchical structure, the power to detect between-group differences depends on the number of populations per group and investigators should sample appropriately.