Solving the spike feature information vanishing problem in spiking deep Q network with potential based normalization.

Area of Science:

• Artificial Intelligence
• Computational Neuroscience

Background:

• Spiking neural networks (SNNs) excel in pattern recognition.
• Deep SNNs show promise in perception but face challenges in reinforcement learning (RL).
• Existing RL-SNN methods often use shallow networks or ANN-SNN conversion.

Purpose of the Study:

• To address the challenge of applying deep SNNs to RL tasks.
• To investigate the signal feature disappearance problem in spiking deep Q networks (SDQN).
• To propose a novel method for direct training of deep SNNs in RL.

Main Methods:

• Mathematical analysis of signal feature disappearance in SDQN.
• Development of a potential-based layer normalization (pbLN) technique.
• Direct training of spiking deep Q networks using the proposed pbLN method (PL-SDQN).

Main Results:

• The proposed PL-SDQN method effectively trains deep SNNs for RL.
• PL-SDQN demonstrates superior performance compared to ANN-SNN conversion methods.
• Experiments on Atari game tasks validate the effectiveness of PL-SDQN.

Conclusions:

• The pbLN method offers a viable solution for training deep SNNs in RL.
• PL-SDQN advances the application of SNNs in complex reinforcement learning domains.
• This work paves the way for more efficient and effective brain-inspired AI in RL.