Conditions for soft selection favoring the evolution of hybrid inviability

Hybrid inviability per se is not generally considered adaptive; it is believed to evolve as a pleiotropic consequence of other genetic changes, either adaptive or neutral, between diverging populations. However, given the pre-existence of hybrid sterility, there are circumstances under which hybrid inviability may be directly selected. In this report, we model the evolution of hybrid inviability in the context of within-family density-dependent selection (soft selection). This paper shows that under certain conditions, an allele causing hybrid inviability can increase when rare, despite also decreasing viability in conspecifics. These conditions depend upon the strength of density effects within families and upon the frequency of matings with heterospecifics. It is found that, under the most favorable conditions (strong soft selection and frequent hybridization), the inviability allele can invade when its deleterious effect on the hybrid viability exceeds four times its deleterious effects on conspecific heterozygotes. The conditions for fixation of the allele are also investigated and the circumstances under which this process might operate in natural populations are discussed. The conclusion is that the most restrictive condition is that for this process to evolve, there must be little opportunity for postzygotic reproductive isolation to evolve.