Population differentiation through mutation and drift--a comparison of genetic identity measures

Different genetic identity or distance measures are compared that consider allelic variation within and between populations. Particularily we analyse those suggested by Nei (IS, DS), Rogers (DR), Reynolds, Weir and Cockerham (D theta), Nei, Tajima and Tateno (DA), Tomiuk and Loeschcke (ITL, DTL) and Goldstein et al. ((delta mu)2). The simulations focus on the influence of non-equilibrium conditions on the stability of these measures. The degree of homozygosity of an ancestral population before it splits into two sister populations is most important for the stability of the different estimates of genetic identity. If populations are not close to their equilibrium homozygosity, a considerable bias can occur and, thereby, provide very misleading estimates of the time span since divergence. The ITL-measure based on estimates of ancestral alleles is more robust than other measures of genetic identity, especially for large population sizes and high mutation rates. For the infinite allele model, the analysis shows that more precise estimates of the frequency of ancestral alleles can greatly improve the reliability of the estimate of genetic identity in the case of ITL. For the stepwise mutation model, the TL-measure combines the attributes of the DA- and (delta mu)2-measures. The TL-measure is efficient for the construction of the correct tree topology of related populations as well as for the estimation of the branch length when protein or microsatellite data are analysed.