Memory length and space shape multi-agent Q-learning dynamics

Area of Science:

• Behavioral Economics
• Game Theory
• Artificial Intelligence

Background:

• In repeated interactions, players adapt strategies based on past moves, necessitating complex cognitive abilities with increased memory.
• Social learning through strategy imitation doesn't guarantee improved payoffs.
• Reinforcement learning models human learning via past experiences, independent of co-players' strategies.

Purpose of the Study:

• To investigate the impact of different memory lengths and spaces (memory-n, reactive-n, reactive-n counting) on cooperation evolution in reinforcement learning agents.
• To determine how memory characteristics influence cooperative behavior and strategy effectiveness.

Main Methods:

• Simulations of multi-agent systems using reinforcement learning agents with varying memory structures.
• Analysis of cooperation levels and strategy evolution under different memory conditions (memory-n, reactive-n, reactive-n counting).

Main Results:

• Memory-n players exhibited higher cooperation rates compared to reactive-n players.
• Increased memory length promoted cooperation in memory-n agents but hindered it in reactive-n agents.
• Reactive-n counting agents demonstrated memory compression, mitigating negative effects of excessive memory.

Conclusions:

• Mutual cooperation and retaliation strategies are crucial for maintaining cooperation in reinforcement learning agents.
• Optimizing the information available to players is essential for promoting cooperation in multi-agent systems.