Single-gene speciation with pleiotropy: effects of allele dominance, population size, and delayed inheritance

Single-gene speciation is considered to be unlikely, but an excellent example is found in land snails, in which a gene for left-right reversal has given rise to new species multiple times. This reversal might be facilitated by their small population sizes and maternal effect (i.e., "delayed inheritance," in which an individual's phenotype is determined by the genotype of its mother). Recent evidence suggests that a pleiotropic effect of the speciation gene on antipredator survival may also promote speciation. Here we theoretically demonstrate that, without a pleiotropic effect, in small populations the fixation probability of a recessive mutant is higher than a dominant mutant, but they are identical for large populations and sufficiently weak selection. With a pleiotropic effect that increases mutant viability, a dominant mutant has a higher fixation probability if the strength of viability selection is sufficiently greater than that of reproductive incompatibility, whereas a recessive mutant has a higher fixation probability otherwise. Delayed inheritance increases the fixation probability of a mutant if viability selection is sufficiently weaker than reproductive incompatibility. Our results clarify the conflicting effects of viability selection and positive frequency-dependent selection due to reproductive incompatibility and provide a new perspective to single-gene speciation theory.