Mutation-selection balance for environmental variance

The role of mutation-selection balance in maintaining environmental variance (V(E)) of quantitative traits is investigated under the assumption that genotypes differ in the magnitude of phenotypic variance, given genotypic value. Thus, V(E) can be regarded as a quantitative trait. As stabilizing selection on phenotype favors genotypes contributing low V(E), mutations that decrease V(E) are more likely to become fixed than those that increase it, and therefore V(E) should decline. If, however, essentially all mutants increase V(E) and overall selection is sufficiently strong that no mutants become fixed, then V(E) can be maintained. The heritability of the trait is determined by the relative sizes of mutational effects on phenotypic mean and residual variance and is independent of mutation rate and pleiotropic effects. This conclusion is not robust for small populations because some mutants may become fixed, which indicates that other selective forces must be involved, such as an intrinsic cost of homogeneity.