Frequency-dependent selection with dominance: a window onto the behavior of the mean fitness

Selection in which fitnesses vary with the changing genetic composition of the population may facilitate the maintenance of genetic diversity in a wide range of organisms. Here, a detailed theoretical investigation is made of a frequency-dependent selection model, in which fitnesses are based on pairwise interactions between the two phenotypes at a diploid, diallelic, autosomal locus with complete dominance. The allele frequency dynamics are fully delimited analytically, along with all possible shapes of the mean fitness function in terms of where it increases or decreases as a function of the current allele frequency in the population. These results in turn allow possibly the first complete characterization of the dynamical behavior by the mean fitness through time under frequency-dependent selection. Here the mean fitness (i) monotonically increases, (ii) monotonically decreases, (iii) initially increases and then decreases, or (iv) initially decreases and then increases as equilibrium is approached. We analytically derive the exact initial and fitness conditions that produce each dynamic and how often each arises. Computer simulations with random initial conditions and fitnesses reveal that the potential decline in mean fitness is not negligible; on average a net decrease occurs 20% of the time and reduces the mean fitness by >17%.