A tiny fraction of all species forms most of nature: Rarity as a sticky state

Using data from a wide range of natural communities including the human microbiome, plants, fish, mushrooms, rodents, beetles, and trees, we show that universally just a few percent of the species account for most of the biomass. This is in line with the classical observation that the vast bulk of biodiversity is very rare. Attempts to find traits allowing the tiny fraction of abundant species to escape rarity have remained unsuccessful. Here, we argue that this might be explained by the fact that hyper-dominance can emerge through stochastic processes. We demonstrate that in neutrally competing groups of species, rarity tends to become a trap if environmental fluctuations result in gains and losses proportional to abundances. This counter-intuitive phenomenon arises because absolute change tends to zero for very small abundances, causing rarity to become a "sticky state", a pseudoattractor that can be revealed numerically in classical ball-in-cup landscapes. As a result, the vast majority of species spend most of their time in rarity leaving space for just a few others to dominate the neutral community. However, fates remain stochastic. Provided that there is some response diversity, roles occasionally shift as stochastic events or natural enemies bring an abundant species down allowing a rare species to rise to dominance. Microbial time series spanning thousands of generations support this prediction. Our results suggest that near-neutrality within niches may allow numerous rare species to persist in the wings of the dominant ones. Stand-ins may serve as insurance when former key species collapse.