Evolution of dominance under frequency-dependent intraspecific competition in an assortatively mating population

We study the evolution of higher levels of dominance as a response to negative frequency-dependent selection. In contrast to previous studies, we focus on the effect of assortative mating on the evolution of dominance under frequency-dependent intraspecific competition. We analyze a two-locus two-allele model, in which the primary locus has a major effect on a quantitative trait that is under a mixture of frequency-independent stabilizing selection, density-dependent selection, and frequency-dependent selection caused by intraspecific competition for a continuum of resources. The second (modifier) locus determines the degree of dominance at the trait level. Additionally, the population mates assortatively with respect to similarities in the ecological trait. Our analysis shows that the parameter region in which dominance can be established decreases if small levels of assortment are introduced. In addition, the degree of dominance that can be established also decreases. In contrast, if assortment is intermediate, sexual selection for extreme types can be established, which leads to evolution of higher levels of dominance than under random mating. For modifiers with large effects, intermediate levels of assortative mating are most favorable for the evolution of dominance. For large modifiers, the speed of fixation can even be higher for intermediate levels of assortative mating than for random mating.