Training Spiking Neural Networks for Reinforcement Learning Tasks With Temporal Coding Method.

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Spiking Neural Networks (SNNs) are increasingly demanded for AI systems.
• Combining SNNs with reinforcement learning (RL) requires effective training methods.
• Temporal coding offers a way to train SNNs while preserving their asynchronous nature.

Purpose of the Study:

• To propose a novel training method for SNNs in RL tasks using temporal coding.
• To address the challenges of spike sparsity and information loss in temporal-coded SNNs.
• To enable SNNs to achieve state-of-the-art performance in RL benchmark tasks.

Main Methods:

• A training method enabling temporal coding in RL tasks for SNNs.
• Introduction of a self-incremental variable to mitigate spike sparsity and ensure differentiability.
• Development of an encoding method to prevent information loss in temporal-coded inputs.

Main Results:

• The proposed method effectively trains SNNs for RL tasks.
• The self-incremental variable successfully pushes spiking neurons to fire, enhancing network activity.
• The encoding method preserves information in temporal-coded inputs.
• Experimental results demonstrate comparable performance to state-of-the-art artificial neural networks on benchmark RL tasks.

Conclusions:

• The proposed training method is effective for SNNs in RL.
• The method overcomes key limitations of temporal coding in SNNs.
• This advancement facilitates the use of SNNs in complex AI applications.