The effect of epistasis on the structure of hybrid zones

Within hybrid zones that are maintained by a balance between selection and dispersal, linkage disequilibrium is generated by the mixing of divergent populations. This linkage disequilibrium causes selection on each locus to act on all other loci, thereby steepening clines, and generating a barrier to gene flow. Diffusion models predict simple relations between the strength of linkage disequilibrium and the dispersal rate, sigma, and between the barrier to gene flow, B, and the reduction in mean fitness, W. The aim of this paper is to test the accuracy of these predictions by comparison with an exact deterministic model of unlinked loci (r = 0.5). Disruptive selection acts on the proportion of alleles from the parental populations (p,q): W = exp[-S(4pq)beta], such that the least fit genotype has fitness e-s. Where beta << 1, fitness is reduced for a wide range of intermediate genotypes; where beta >> 1, fitness is only reduced for those genotypes close to p = 0.5. Even with strong epistasis, linkage disequilibria are close to sigma 2p'ip'j/rij, where p'i, p'j are the gradients in allele frequency at loci i, j. The barrier to gene flow, which is reflected in the steepening of neutral clines, is given by [formula: see text] where r, the harmonic mean recombination rate between the neural and selected loci, is here 0.5. This is a close approximation for weak selection, but underestimates B for strong selection. The barrier is stronger for small beta, because hybrid fitness is then reduced over a wider range of p. The widths of the selected clines are harder to predict: though simple approximations are accurate for beta = 1, they become inaccurate for extreme beta because, then, fitness changes sharply with p. Estimates of gene number, made from neutral clines on the assumption that selection acts against heterozygotes, are accurate for weak selection when beta = 1; however, for strong selection, gene number is overestimated. For beta > 1, gene number is systematically overestimated and, conversely, when beta < 1, it is underestimated.