Reinforcement learning with reputation-based adaptive exploration promotes cooperation

Area of Science:

• Computational Social Science
• Behavioral Economics
• Artificial Intelligence

Background:

• Reinforcement learning models behavior adjustment via feedback.
• Q-learning uses exploration rates, typically constant, to balance action selection.
• Social evaluation systems imply exploration costs/benefits vary with reputation.

Purpose of the Study:

• To develop a spatial prisoner's dilemma model where Q-learning agents adapt exploration rates based on local reputation.
• To investigate how asymmetric, state-dependent reputation updating influences cooperation.
• To analyze the combined effects of adaptive exploration and reputation dynamics on cooperation.

Main Methods:

• Developed a spatial prisoner's dilemma model with Q-learning agents.
• Implemented adaptive exploration rates dependent on local reputation differences.
• Utilized an asymmetric, state-dependent rule for reputation updates.
• Analyzed spatial organization and stability of cooperation.

Main Results:

• Adaptive exploration and asymmetric reputation updating individually promote cooperation.
• Their combination yields a stronger cooperative effect than either mechanism alone.
• Low-reputation agents increase exploration to recover reputation; high-reputation agents decrease it to avoid losses.
• A stable checkerboard pattern of cooperation emerges at intermediate reputation concern.
• Stronger asymmetric reputation updating mitigates cooperation disruption caused by intermediate exploration rates.

Conclusions:

• Reputation dynamically regulates exploratory behavior during learning, stabilizing cooperation.
• Adaptive exploration based on reputation enhances cooperation more than fixed exploration rates.
• The model demonstrates how social reputation can foster prosocial behavior in learning agents.