Revisiting FIS, FST, Wahlund Effects, and Null Alleles

Null alleles and Wahlund effects are well known causes of heterozygote deficits in empirical population genetics studies as compared to Hardy-Weinberg genotypic expectations. Some authors have theoretically studied the relationship of Wright's FIS computed from subsamples displaying a Wahlund effect and FST before the Wahlund effect, as can occasionally be obtained from populations of long-lived organisms. In the 2 subsample case, a positive relationship between these 2 parameters across loci would represent a signature of Wahlund effects. Nevertheless, for most organisms, getting 2 independent subsamples of the same cohort and population, one with a Wahlund effect and the other without, is almost never achieved and most of the time, empirical population geneticists only collect a single sample, with or without a Wahlund effect, or with or without null alleles. Another issue is that null allele increase FIS and FST altogether and thus may also create such correlation. In this article, I show that, for organisms collected in a single sample, which corresponds to the most common situation, Wahlund effects and null alleles affect the values of both FIS and FST though in the opposite direction. I also show that Wahlund effect produces no or weak positive correlation between the 2 F-statistics, while null alleles generate a strong positive correlation between them. Variation of these F-statistics is small and even minimized for FST under Wahlund effects as compared to null alleles. I finally propose a determination key to interpret data with heterozygote deficits.