Deviations from Hardy-Weinberg proportions: sampling variances and use in estimation of inbreeding coefficients

An analysis is made of the distribution of deviations from Hardy-Weinberg proportions with k alleles and of estimates of inbreeding coefficients (f) obtained from these deviations. If f is small, the best estimate of f in large samples is shown to be 2 sigma i(Tii/Ni)/(k - 1), where Tii is an unbiased measure of the excess of the ith homozygote and Ni the number of the ith allele in the sample [frequency = Ni/(2N)]. No extra information is obtained from the Tij, where these are departures of numbers of heterozygotes from expectation. Alternatively, the best estimator can be computed from the Tij, ignoring the Tii. Also (1) the variance of the estimate of f equals 1/(N(k - 1] when all individuals in the sample are unrelated, and the test for f = 0 with 1 d.f. is given by the ratio of the estimate to its standard error; (2) the variance is reduced if some alleles are rare; and (3) if the sample consists of full-sib families of size n, the variance is increased by a proportion (n - 1)/4 but is not increased by a half-sib relationship. If f is not small, the structure of the population is of critical importance. (1) If the inbreeding is due to a proportion of inbred matings in an otherwise random-breeding population, f as determined from homozygote excess is the same for all genes and expressions are given for its sampling variance. (2) If the homozygote excess is due to population admixture, f is not the same for all genes. The above estimator is probably close to the best for all f values.