Looking for the right mate in diploid species: How does genetic dominance affect the spatial differentiation of a sexual trait?

Divergence between populations for a given trait can be driven by sexual selection, interacting with migration behaviour. Mating preference for different phenotypes may lead to specific migration behaviour, with departures from populations where the preferred trait is rare. Such preferences can then trigger the emergence and persistence of differentiated populations, even without any local adaptation. However the genetic architecture underlying the trait targeted by mating preference may have a profound impact on population divergence. In particular, dominance between alleles encoding for divergent phenotypes can interfere with the differentiation process. Using a diploid model of a trait determining both mating success and migration rate, we explored differentiation between two connected populations, assuming either co-dominance or strict dominance between alleles. The model assumes that individuals prefer mating with partners displaying the same phenotype and therefore tend to move to the other population when their own phenotype is rare. We show that the emergence of differentiated populations in this diploid moded is limited as compared to results obtained with the same model assuming haploidy. When assuming co-dominance, differentiation arises only when migration is limited compared to the strength of the preference. Such differentiation is less dependent on migration when assuming strict dominance between haplotypes. Dominant alleles frequently invade populations because their phenotype is more frequently expressed, resulting in higher local mating success and a rapid decrease in migration. However, depending on the initial distribution of alleles, this advantage associated with dominance (i.e. Haldane's sieve) may lead to fixation of the dominant allele throughout both populations. Depending on the initial distribution of heterozygotes in the two populations, persistence of polymorphisms within populations can also occur because heterozygotes displaying the predominant phenotype benefit from high mating success. Altogether, our results highlight that heterozygotes' behaviour has a strong impact on population differentiation and highlight the need for diploid models of differentiation and speciation driven by sexual selection.