Reinforcement learning to train a cooperative network with both discrete and continuous output neurons

We propose a reinforcement learning algorithm to train a cooperative network with both discrete and continuous output neurons based on the finding that discrete and continuous motorneurons coexist in the gill-withdrawal neural network of the sea mollusk, Aplysia. The network was trained to control an inverted pendulum. Simulation experiments showed that the two output neurons had distinct but cooperative roles: the discrete output neuron was essential for fast learning while the continuous output neuron was necessary for learning fine control. To achieve both fast learning and fine control, the shape of the sigmoid function in the continuous output neuron should be set before learning.