Detecting dominant QTL with variance component analysis in simulated pedigrees

Dominance is an important source of variation in complex traits. Here, we have carried out the first thorough investigation of quantitative trait locus (QTL) detection using variance component (VC) models extended to incorporate both additive and dominant QTL effects. Simulation results showed that the empirical distribution of the test statistic when testing for dominant QTL effects did not behave in accordance with existing theoretical expectations and varied with pedigree structure. Extensive simulations were carried out to assess accuracy of estimates, type 1 error and statistical power in two-generation human-, poultry- and pig-type pedigrees each with 1900 progeny in small-, medium- and large-sized families, respectively. The distribution of the likelihood-ratio test statistic was heavily dependent on family structure, with empirical thresholds lower for human pedigrees. Power to detect QTL was high (0.84-1.0) in pig and poultry scenarios for dominance effects accounting for >7% of phenotypic variance but much lower (0.42) in human-type pedigrees. Maternal or common environment effects can be partially confounded with dominance and must be fitted in the QTL model. Including dominance in the QTL model did not affect power to detect additive QTL effects. Also, detection of spurious dominance QTL effects only occurred when maternal effects were not included in the QTL model. When dominance effects were present in the data but not in the analysis model, this resulted in spurious detection of additive QTL or inflated estimates of additive QTL effects. The study demonstrates that dominance can be included routinely in QTL analysis of general pedigrees; however, optimal power is dependent on selection of the appropriate thresholds for pedigree structure.