A classification of structured-coalescent processes with migration, conditional on the population pedigree

Organismal genealogies or pedigrees of populations may have strong effects on patterns of variation. We extend recent analyses of the effects that pedigrees have on times to common ancestry to cases of population subdivision and migration. These analyses condition on the population pedigree, while traditional coalescent models marginalize over it. A finding of a "pedigree effect" implies that data analysis and interpretation should not be based on the corresponding traditional coalescent model but rather on a coalescent model obtained by conditioning on the pedigree. We apply a straightforward test based on the distribution of pairwise coalescence times to four previously described scenarios of subdivision and migration. These scenarios are defined by the relative magnitudes of four parameters: the number of local populations or demes, the deme size, the migration fraction, and the probability that migration can occur at all. We find pedigree effects in three scenarios. In two of the three, the effect is weak if the deme size is large. The one scenario without any pedigree effect corresponds to the well-known structured-coalescent model. The one scenario with a persistent pedigree effect even in the limit as the deme size tends to infinity involves long periods without gene flow interrupted by pulses of migration. We illustrate our results using simulations and numerical analysis. The implications of our results for statistical genetic inference in structured populations include that the structured-coalescent model may be applied to multiple unlinked loci, and that pedigrees will need to be accounted for when deme sizes are not large.