Reinforcement-learning-based replicator-mutator dynamics for game-environment coevolution in public goods games

Area of Science:

• Mathematical Biology
• Evolutionary Game Theory
• Ecology

Background:

• Existing models often overlook long-term learning in strategy adaptation.
• Game-environment feedback dynamics are crucial in ecological and biological systems.

Purpose of the Study:

• To model how long-term learning influences strategy adjustment.
• To develop reinforcement-learning-based dynamics for game-environment feedback.

Main Methods:

• Derived a learning-induced mutation term from Q-learning.
• Embedded this term into replicator-mutator dynamics for public goods games.
• Analyzed the impact of learning rates on system dynamics.

Main Results:

• The model captures nonlinear, frequency-dependent strategy adjustments based on long-term learning.
• Learning rates significantly influence cooperation levels and environmental states.
• Appropriate learning rates can mitigate the tragedy of the commons.
• Low learning rates can lead to damped oscillations in strategy-environment dynamics.

Conclusions:

• Long-term learning, modeled via Q-learning, is essential for understanding strategy evolution in changing environments.
• The derived dynamics offer insights into regulating coevolutionary fluctuations through learning-induced exploration.
• This framework helps explain how cooperation can be sustained in ecological systems.