Extreme patterns of variance in small populations: placing limits on human Y-chromosome diversity through time in the...

Small populations are dominated by unique patterns of variance, largely characterized by rapid drift of allele frequencies. Although the variance components of genetic datasets have long been recognized, most population genetic studies still treat all sampling locations equally despite differences in sampling and effective population sizes. Because excluding the effects of variance can lead to significant biases in historical reconstruction, variance components should be incorporated explicitly into population genetic analyses. The possible magnitude of variance effects in small populations is illustrated here via a case study of Y-chromosome haplogroup diversity in the Vanuatu Archipelago. Deme-based modelling is used to simulate allele frequencies through time, and conservative confidence bounds are placed on the accumulation of stochastic variance effects, including diachronic genetic drift and contemporary sampling error. When the information content of the dataset has been ascertained, demographic models with parameters falling outside the confidence bounds of the variance components can then be accepted with some statistical confidence. Here I emphasize how aspects of the demographic history of a population can be disentangled from stochastic variance effects, and I illustrate the extreme roles of genetic drift and sampling error for many small human population datasets.