Deleterious mutations, variable epistatic interactions, and the evolution of recombination

In this paper, we examine the conditions that allow increased recombination to evolve in the presence of recurrent deleterious mutation. We focus on a three-locus model first studied by Feldman et al. (1980), which follows the dynamics of a modifier locus that alters the recombination rate between two loci subject to deleterious mutation. Although Feldman et al. (1980) indicated that increased recombination might be favored if there is diminishing-returns epistasis, we show that alleles that increase the recombination rate can only invade if there is synergistic epistasis between the loci under selection. Even with synergistic epistasis, evolution at the modifier locus will lead to decreased recombination if the modifier locus is loosely linked and epistasis is strong. Using the multi-locus analysis of Barton (1995), we show that variability among loci in the sign and strength of epistasis further decreases the parameter space over which increased recombination may evolve. We conclude that, even with negative epistasis, increased recombination may only be favored when linkage is tight, especially if, as seems likely, epistatic interactions are highly variable among loci.