Measwring and Testing Genetic Differentiation With Ordered <i>Versus</i> Unordered Alleles

TLDR

NST and GST differences arise from sampling artefacts, mutation‑rate effects, and phylogeographic structure. The study proposes estimates and variances of diversity and differentiation measures for subdivided populations that apply to haplotypes. The authors develop a method that compares GST (allele‑frequency based) and NST (haplotype‑similarity based), tests these against zero and each other, and applies it to a grasshopper nuclear DNA dataset. Applying the method to the grasshopper data, progressively merging related haplotypes revealed additional insights into population genetic subdivision.

Abstract

Abstract Estimates and variances of diversity and differentiation measures in subdivided populations are proposed that can be applied to haplotypes (ordered alleles such as DNA sequences, which may contain a record of their own histories). Hence, two measures of differentiation can be compared for a single data set: one (GST) that makes use only of the allelic frequencies and the other (NST) for which similarities between the haplotypes are taken into account in addition. Tests are proposed to compare NST and GST with zero and with each other. The difference between NST and GST can be caused by several factors, including sampling artefacts, unequal effect of mutation rates and phylogeographic structure. The method presented is applied to a published data set where a nuclear DNA sequence had been determined from individuals of a grasshopper distributed in 24 regions of Europe. Additional insights into the genetic subdivision of these populations are obtained by progressively combining related haplotypes and reanalyzing the data each time.