Spike-based decision learning of Nash equilibria in two-player games

Area of Science:

• Computational Neuroscience
• Game Theory
• Machine Learning

Background:

• Decision-making in uncertain, multi-agent environments is complex due to co-adaptive strategies.
• The neural basis and computational algorithms for adaptive decision-making remain largely unknown.

Purpose of the Study:

• To propose and validate a population coding model of spiking neurons for adaptive decision-making.
• To investigate the model's ability to acquire optimal strategies in game-theoretical tasks and reproduce human behavioral data.

Main Methods:

• Developed a population reinforcement learning model using spiking neurons and a policy gradient procedure.
• Tested the model on classical game-theoretical tasks, including blackjack and the inspector game.
• Compared the model's performance against temporal-difference (TD)-learning, covariance-learning, and basic reinforcement learning.

Main Results:

• The proposed model successfully acquired optimal strategies for both deterministic and stochastic Nash equilibria.
• Population reinforcement learning reproduced human behavioral data in blackjack and inspector games.
• Other learning methods failed to achieve optimal performance for stochastic strategies.

Conclusions:

• Spiking neural network-based population reinforcement learning provides a viable computational framework for understanding automated decision learning of Nash equilibria.
• The model's ability to follow the stochastic reward gradient supports its role in explaining adaptive decision-making in two-player games.