Reinforcement Learning With Low-Complexity Liquid State Machines.

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Machine Learning

Background:

• Deep reinforcement learning models often require extensive trainable parameters and computational resources.
• Spiking neural networks (SNNs) offer a biologically plausible and potentially more efficient alternative for complex computations.

Purpose of the Study:

• To develop a computationally efficient reinforcement learning framework using small, randomly connected spiking neural networks.
• To demonstrate that these networks can achieve high learning efficiency comparable to humans on complex tasks.

Main Methods:

• Proposing reinforcement learning on sparse, randomly interconnected recurrent and feed-forward spiking neural networks.
• Utilizing systematic initialization of random connections and training a readout layer with the Q-learning algorithm.
• Leveraging the non-linear dynamics of SNNs for rich, high-dimensional input representations.

Main Results:

• The proposed small random spiking networks learn complex reinforcement learning tasks with very few trainable parameters.
• These networks exhibit efficient temporal integration and learn effectively even with partial state inputs due to fading memory.
• Achieved learning efficiency comparable to human performance on tasks like Atari games.

Conclusions:

• Small random spiking networks provide a computationally efficient and effective alternative to deep reinforcement learning architectures.
• The sparse, recurrent nature of these networks facilitates context-dependent processing and memory retention.
• This approach demonstrates the potential of biologically inspired neural network models for advanced AI applications.