On the instability of evolutionary stable strategies

Evolutionary stable strategies (ESSs) are often used to explain the behaviors of individuals and species. The analysis of ESSs determines which, if any, combinations of behaviors cannot be invaded by alternative strategies. However, two of the assumptions required to generate ESSs, an infinite population and payoffs described only on the average, are not particularly realistic in natural situations. Previous experiments have indicated that under more natural conditions of finite populations and stochastic payoffs, populations may evolve in trajectories that are unrelated to an ESS, even in very simple evolutionary games. Those earlier simulations are extended here under a variety of conditions. The results suggest that ESSs may not provide a good explanation of a finite population's behavior even when the conditions correspond closely with the infinite population model. The implications of these results are discussed briefly in light of previous literature claiming that ESSs generated suitable explanations of real-world data.