Genetic Differentiation and Estimation of Gene Flow from <i>F</i>-Statistics Under Isolation by Distance

TLDR

Regression of FST/(1‑FST) on geographic distance can estimate the product of population density and the second moment of parental axial distance. The study reexamines isolation‑by‑distance models for estimating demographic parameters from population subdivision measures and refines F‑statistic and coalescence time results in one‑ and two‑dimensional settings. The authors propose regressing FST/(1‑FST) on geographic distance (or its logarithm in two dimensions) to estimate demographic parameters. In two cases where direct comparison is possible, the proposed method outperforms previous indirect methods.

Abstract

I reexamine the use of isolation by distance models as a basis for the estimation of demographic parameters from measures of population subdivision. To that aim, I first provide results for values of F-statistics in one-dimensional models and coalescence times in two-dimensional models, and make more precise earlier results for F-statistics in two-dimensional models and coalescence times in one-dimensional models. Based on these results, I propose a method of data analysis involving the regression of FST/(1-FST) estimates for pairs of subpopulations on geographic distance for populations along linear habitats or logarithm of distance for populations in two-dimensional habitats. This regression provides in principle an estimate of the product of population density and second moment of parental axial distance. In two cases where comparison to direct estimates is possible, the method proposed here is more satisfactory than previous indirect methods.