The mapping of epistatic effects onto a genealogical tree in haploid populations

In this paper we present a model that maps epistatic effects onto a genealogical tree for a haploid population. Prior work has demonstrated that genealogical structure causes the genotypic values of individuals to covary. Our results indicate that epistasis can reduce genotypic covariance that is caused by genealogical structure. Genotypic effects (both additive and epistatic) occur along the branches of a genealogical tree, from the base of the tree to its tips. Epistasis reduces genotypic covariance because there is a reweighting of the contribution of branches to the states of genotypes compared to the additive case. Branches near the tips of a genealogical tree contribute proportionally more genetic effects with epistasis than without epistasis. Epistatic effects are most numerous at basal positions in a genealogical tree when a population is constant in size and experiencing no selection, optimizing selection, diversifying selection or directional selection, indicating that epistatic effects are typically old. For a population that is growing in size, epistatic effects are most numerous at midpoints in a genealogical tree, indicating epistatic effects are of moderate age. Our results are important in that they suggest epistatic effects may typically explain deep (old) divergences and broad patterns of divergence that exist in populations, except in growing populations. In a growing population, epistatic effects may cause more within group divergence higher up in a tree and less between group divergence that is deep in a tree. The distribution of the number of epistatic effects and the expected variance and covariance in the number of epistatic effects is also provided assuming neutrality.