Epistasis of quantitative trait loci under different gene action models

Modeling and detecting nonallelic (epistatic) effects at multiple quantitative trait loci (QTL) often assume that the study population is in zygotic equilibrium (i.e., genotypic frequencies at different loci are products of corresponding single-locus genotypic frequencies). However, zygotic associations can arise from physical linkages between different loci or from many evolutionary and demographic processes even for unlinked loci. We describe a new model that partitions the two-locus genotypic values in a zygotic disequilibrium population into equilibrium and residual portions. The residual portion is of course due to the presence of zygotic associations. The equilibrium portion has eight components including epistatic effects that can be defined under three commonly used equilibrium models, Cockerham's model, F2-metric, and F(infinity)-metric models. We evaluate our model along with these equilibrium models theoretically and empirically. While all the equilibrium models require zygotic equilibrium, Cockerham's model is the most general, allowing for Hardy-Weinberg disequilibrium and arbitrary gene frequencies at individual loci whereas F2-metric and F(infinity)-metric models require gene frequencies of one-half in a Hardy-Weinberg equilibrium population. In an F2 population with two unlinked loci, Cockerham's model is reduced to the F2-metric model and thus both have a desirable property of orthogonality among the genic effects; the genic effects under the F(infinity)-metric model are not orthogonal but they can be easily translated into those under the F2-metric model through a simple relation. Our model is reduced to these equilibrium models in the absence of zygotic associations. The results from our empirical analysis suggest that the residual genetic variance arising from zygotic associations can be substantial and may be an important source of bias in QTL mapping studies.