Democratic population decisions result in robust policy-gradient learning: a parametric study with GPU simulations.

Area of Science:

• Computational Neuroscience
• High-Performance Computing
• Artificial Neural Networks

Background:

• Graphics Processing Units (GPUs) offer high computational power for complex simulations.
• GPU programming presents challenges, necessitating investigation into its utility for neuroscience.
• Simulating spiking neural networks requires significant computational resources.

Purpose of the Study:

• To advocate for GPU utilization in Computational Neuroscience.
• To explore optimal conditions for a two-layer neural network's performance using GPU computing.
• To assess the impact of recurrent connectivity on learning simplified navigation tasks.

Main Methods:

• Simulated a two-layer network of Integrate-and-Fire neurons using GPU programming.
• Implemented a policy-gradient learning rule from Reinforcement Learning.
• Investigated varying degrees of recurrent connectivity, including Mexican-Hat-shaped connections and their absence.

Main Results:

• Networks with strong Mexican-Hat recurrent connections showed mediocre learning results.
• Networks without recurrent connections, utilizing a "democratic" population vector readout, learned the task more robustly.
• GPU programming achieved a 5x to 42x speed improvement over optimized Python code, especially in parameter search.

Conclusions:

• The absence of specific recurrent connections enhances learning performance in this neural network architecture.
• Efficient GPU programming significantly reduces simulation time for spiking neural networks, particularly for parameter exploration.
• GPUs are valuable tools for advancing research in Computational Neuroscience.