The joint effects of selection and dominance on the QST - FST contrast

Q(ST) measures the differentiation of quantitative traits between populations. It is often compared to F(ST), which measures population differentiation at neutral marker loci due to drift, migration, and mutation. When Q(ST) is different from F(ST), it is usually taken as evidence that selection has either restrained or accelerated the differentiation of the quantitative trait relative to neutral markers. However, a number of other factors such as inbreeding, dominance, and epistasis may also affect the Q(ST) - F(ST) contrast. In this study, we examine the effects of dominance, selection, and inbreeding on Q(ST) - F(ST). We compare Q(ST) with F(ST) at selected and neutral loci for populations at equilibrium between selection, drift, mutation, and migration using both analytic and simulation approaches. Interestingly, when divergent selection is acting on a locus, inbreeding and dominance generally inflate Q(ST) relative to F(ST) when they are both measured at the quantitative locus at equilibrium. As a consequence, dominance is unlikely to hide the signature of divergent selection on the Q(ST) - F(ST) contrast. However, although in theory dominance and inbreeding affect the expectation for Q(ST) - F(ST), of most concern is the very large variance in both Q(ST) and F(ST), suggesting that we should be cautious in attributing small differences between Q(ST) and F(ST) to selection.