Reinforcement learning to train a cooperative network with both discrete and continuous output neurons

Area of Science:

• Neuroscience
• Computational Neuroscience
• Machine Learning

Background:

• The sea mollusk Aplysia exhibits a neural network with both discrete and continuous motor neurons controlling gill withdrawal.
• Understanding neural control mechanisms can inform the development of advanced AI algorithms.

Purpose of the Study:

• To propose a novel reinforcement learning algorithm for training cooperative networks with mixed discrete and continuous output neurons.
• To investigate the distinct roles of discrete and continuous neurons in learning complex tasks.

Main Methods:

• A reinforcement learning algorithm was designed to train a cooperative network.
• The network, featuring discrete and continuous output neurons, was trained to control an inverted pendulum.
• Simulation experiments were conducted to analyze neuron roles and optimal parameter settings.

Main Results:

• Discrete and continuous output neurons demonstrated distinct yet cooperative functions.
• Discrete neurons were crucial for rapid learning, while continuous neurons facilitated fine control.
• Pre-setting the sigmoid function's shape in continuous neurons was found essential for achieving both fast learning and fine control.

Conclusions:

• Cooperative networks with mixed discrete and continuous outputs can effectively learn complex tasks.
• The specific roles of discrete and continuous neurons are vital for optimizing learning speed and precision.
• Optimizing network parameters, such as sigmoid function shape, prior to learning is key for balanced performance.