Inclusive fitness and Hamilton's rule in a stochastic environment

The evolution of cooperation in Prisoner's Dilemmas with additive random cost and benefit for cooperation cannot be accounted for by Hamilton's rule based on mean effects transferred from recipients to donors weighted by coefficients of relatedness, which defines inclusive fitness in a constant environment. Extensions that involve higher moments of stochastic effects are possible, however, and these are connected to a concept of random inclusive fitness that is frequency-dependent. This is shown in the setting of pairwise interactions in a haploid population with the same coefficient of relatedness between interacting players. In an infinite population, fixation of cooperation is stochastically stable if a mean geometric inclusive fitness of defection when rare is negative, while fixation of defection is stochastically unstable if a mean geometric inclusive fitness of cooperation when rare is positive, and these conditions are generally not equivalent. In a finite population, the probability for cooperation to ultimately fix when represented once exceeds the probability under neutrality or the corresponding probability for defection if the mean inclusive fitness of cooperation when its frequency is 1/3 or 1/2, respectively, exceeds 1. All these results rely on the simplifying assumption of a linear fitness function. It is argued that meaningful applications of random inclusive fitness in complex settings (multi-player game, diploidy, population structure) would generally require conditions of weak selection and additive gene action.