The role of deleterious mutations in allopatric speciation

Allopatric speciation is often assumed to occur as a consequence of adaptive divergence between two isolated populations. However, there are some scenarios in which reproductive isolation can be favored due to accumulated unconditionally deleterious mutations. If deleterious mutations have synergistic epistatic effects, it is shown here that the average fitness of recombinants between two parental lines with a given number of fixed mutations is lower than that of the parents in both the F1 and F2 generations. If individual mutations are only slightly deleterious, then they will tend to fixation at a high enough rate to cause lower hybrid fitness. If the fitness effects of mutation give rise to antagonistic epistasis, the hybrids tend to have a higher average fitness than the parental lines, suggesting a possible scenario for the origin of hybrid vigor. The other model of deleterious mutations investigated is the accumulation of knockout mutants in a duplicated gene family. While neutral in the parental lines, upon contact the F1 and later generations have a significant probability of carrying double knockouts. Under this scenario, selection may also favor reproductive isolation between the two lines. Even when the selection coefficients generated are too low to drive speciation, epistatic interactions between deleterious mutations offer a possible explanation for both outbreeding depression and hybrid vigor.