Microsatellite allele sizes: a simple test to assess their significance on genetic differentiation

The mutation process at microsatellite loci typically occurs at high rates and with stepwise changes in allele sizes, features that may introduce bias when using classical measures of population differentiation based on allele identity (e.g., F(ST), Nei's Ds genetic distance). Allele size-based measures of differentiation, assuming a stepwise mutation process [e.g., Slatkin's R(ST), Goldstein et al.'s (deltamu)(2)], may better reflect differentiation at microsatellite loci, but they suffer high sampling variance. The relative efficiency of allele size- vs. allele identity-based statistics depends on the relative contributions of mutations vs. drift to population differentiation. We present a simple test based on a randomization procedure of allele sizes to determine whether stepwise-like mutations contributed to genetic differentiation. This test can be applied to any microsatellite data set designed to assess population differentiation and can be interpreted as testing whether F(ST) = R(ST). Computer simulations show that the test efficiently identifies which of F(ST) or R(ST) estimates has the lowest mean square error. A significant test, implying that R(ST) performs better than F(ST), is obtained when the mutation rate, mu, for a stepwise mutation process is (a) >/= m in an island model (m being the migration rate among populations) or (b) >/= 1/t in the case of isolated populations (t being the number of generations since population divergence). The test also informs on the efficiency of other statistics used in phylogenetical reconstruction [e.g., Ds and (deltamu)(2)], a nonsignificant test meaning that allele identity-based statistics perform better than allele size-based ones. This test can also provide insights into the evolutionary history of populations, revealing, for example, phylogeographic patterns, as illustrated by applying it on three published data sets.