Evolution to alternative levels of stable diversity leaves areas of niche space unexplored

One of the oldest and most persistent questions in ecology and evolution is whether natural communities tend to evolve toward saturation and maximal diversity. Robert MacArthur's classical theory of niche packing and the theory of adaptive radiations both imply that populations will diversify and fully partition any available niche space. However, the saturation of natural populations is still very much an open area of debate and investigation. Additionally, recent evolutionary theory suggests the existence of alternative evolutionary stable states (ESSs), which implies that some stable communities may not be fully saturated. Using models with classical Lotka-Volterra ecological dynamics and three formulations of evolutionary dynamics (a model using adaptive dynamics, an individual-based model, and a partial differential equation model), we show that following an adaptive radiation, communities can often get stuck in low diversity states when limited by mutations of small phenotypic effect. These low diversity metastable states can also be maintained by limited resources and finite population sizes. When small mutations and finite populations are considered together, it is clear that despite the presence of higher-diversity stable states, natural populations are likely not fully saturating their environment and leaving potential niche space unfilled. Additionally, within-species variation can further reduce community diversity from levels predicted by models that assume species-level homogeneity.