Generalization of the Q_ST framework in hierarchically structured populations: Impacts of inbreeding and dominance

Q_ST is a differentiation parameter based on the decomposition of the genetic variance of a trait. In the case of additive inheritance and absence of selection, it is analogous to the genic differentiation measured on individual loci, F_ST . Thus, Q_ST -F_ST comparison is used to infer selection: selective divergence when Q_ST  > F_ST , or convergence when Q_ST  < F_ST. The definition of Q-statistics was extended to two-level hierarchical population structures with Hardy-Weinberg equilibrium. Here, we generalize the Q-statistics framework to any hierarchical population structure. First, we developed the analytical definition of hierarchical Q-statistics for populations not at Hardy-Weinberg equilibrium. We show that the Q-statistics values obtained with the Hardy-Weinberg definition are lower than their corresponding F-statistics when F_IS  > 0 (higher when F_IS  < 0). Then, we used an island model simulation approach to investigate the impact of inbreeding and dominance on the Q_ST -F_ST framework in a hierarchical population structure. We show that, while differentiation at the lower hierarchical level (Q_SR ) is a monotonic function of migration, differentiation at the upper level (Q_RT ) is not. In the case of additive inheritance, we show that inbreeding inflates the variance of Q_RT , which can increase the frequency of Q_RT  > F_RT cases. We also show that dominance drastically reduces Q-statistics below F-statistics for any level of the hierarchy. Therefore, high values of Q-statistics are good indicators of selection, but low values are not in the case of dominance.