Population structure and covariate analysis based on pairwise microsatellite allele matching frequencies

We describe a general model for pairwise microsatellite allele matching probabilities. The model can be used for analysis of population substructure, and is particularly focused on relating genetic correlation to measurable covariates. The approach is intended for cases when the existence of subpopulations is uncertain and a priori assignment of samples to hypothesized subpopulations is difficult. Such a situation arises, for example, with western Arctic bowhead whales, where genetic samples are available only from a possibly mixed migratory assemblage. We estimate genetic structure associated with spatial, temporal, or other variables that may confound the detection of population structure. In the bowhead case, the model permits detection of genetic patterns associated with a temporally pulsed multi-population assemblage in the annual migration. Hypothesis tests for population substructure and for covariate effects can be carried out using permutation methods. Simulated and real examples illustrate the effectiveness and reliability of the approach and enable comparisons with other familiar approaches. Analysis of the bowhead data finds no evidence for two temporally pulsed subpopulations using the best available data, although a significant pattern found by other researchers using preliminary data is also confirmed here. Code in the R language is available from www.stat.colostate.edu/~geof/gammmp.html.