On indirect genetic effects in structured populations

Indirect genetic effects (IGEs) occur when the phenotype of an individual, and possibly its fitness, depends, at least in part, on the genes of its social partners. The effective result is that environmental sources of phenotypic variance can themselves evolve. Simple models have shown that IGEs can alter the rate and direction of evolution for traits involved in interactions. Here we expand the applicability of the theory of IGEs to evolution in metapopulations by including nonlinear interactions between individuals and population genetic structure. Although population subdivision alone generates some dramatic and nonintuitive evolutionary dynamics for interacting phenotypes, the combination of nonlinear interactions with subdivision reveals an even greater importance of IGEs. The presence of genetic structure links the evolution of interacting phenotypes and the traits that influence their expression ("effector traits") even in the absence of genetic correlations. When nonlinear social effects occur in subdivided populations, evolutionary response is altered and can even oppose the direction expected due to direct selection. Because population genetic structure allows for multilevel selection, we also investigate the role of IGEs in determining the response to individual and group selection. We find that nonlinear social effects can cause interference between levels of selection even when they act in the same direction. In some cases, interference can be so extreme that the actual evolutionary response to multilevel selection is opposite in direction to that predicted by summing selection at each level. This theoretical result confirms empirical data that show higher levels of selection cannot be ignored even when selection acts in the same direction at all levels.