When rarity has costs: coexistence under positive frequency-dependence and environmental stochasticity

Stable coexistence relies on negative frequency-dependence, in which rarer species invading a patch benefit from a lack of conspecific competition experienced by residents. In nature, however, rarity can have costs, resulting in positive frequency-dependence (PFD) particularly when species are rare. Many processes can cause positive frequency-dependence, including a lack of mates, mutualist interactions, and reproductive interference from heterospecifics. When species become rare in the community, positive frequency-dependence creates vulnerability to extinction, if frequencies drop below certain thresholds. For example, environmental fluctuations can drive species to low frequencies where they are then vulnerable to PFD. Here, we analyze deterministic and stochastic mathematical models of two species interacting through both PFD and resource competition in a Chessonian framework. Reproductive success of individuals in these models is reduced by a product of two terms: the reduction in fecundity due to PFD, and the reduction in fecundity due to competition. Consistent with classical coexistence theory, the effect of competition on individual reproductive success exhibits negative frequency-dependence when individuals experience greater intraspecific competition than interspecific competition, i.e., niche overlap is less than one. In the absence of environmental fluctuations, our analysis reveals that (1) a synergistic effect of PFD and niche overlap that hastens exclusion, (2) trade-offs between susceptibility to PFD and maximal fecundity can mediate coexistence, and (3) coexistence, when it occurs, requires that neither species is initially rare. Analysis of the stochastic model highlights that environmental fluctuations, unless perfectly correlated, coupled with PFD ultimately drive one species extinct. Over any given time frame, this extinction risk decreases with the correlation of the demographic responses of the two species to the environmental fluctuations, and increases with the temporal autocorrelation of these fluctuations. For species with overlapping generations, these trends in extinction risk persist despite the strength of the storage effect decreasing with correlated demographic responses and increasing with temporal autocorrelations. These results highlight how the presence of PFD may alter the outcomes predicted by modern coexistence mechanisms.