A neural model of hierarchical reinforcement learning

Area of Science:

• Computational Neuroscience
• Cognitive Science
• Artificial Intelligence

Background:

• Reinforcement learning (RL) models in neuroscience face challenges with complex tasks.
• Existing models often lack detailed biological plausibility.
• Hierarchical reinforcement learning (HRL) offers a framework for complex decision-making but requires robust neural implementation.

Purpose of the Study:

• To develop a novel, biologically detailed neural model of reinforcement learning (RL).
• To incorporate complex features like extended action sequences, continuous environments, and noisy computations.
• To expand neural RL into hierarchical reinforcement learning (HRL) with anatomical and physiological constraints.

Main Methods:

• Developed a neural network model integrating detailed biological features.
• Implemented hierarchical reinforcement learning (HRL) components within the neural architecture.
• Tested the model across diverse environments to assess general learning abilities.

Main Results:

• The model successfully handles temporally extended actions, continuous environments, and noisy computations.
• The hierarchical implementation significantly improved performance compared to non-hierarchical models.
• Model behavior aligns with human HRL data, generating new predictions.

Conclusions:

• This biologically detailed neural model provides a robust framework for understanding hierarchical reinforcement learning in the brain.
• The model's success in complex environments and consistency with human data validate its approach.
• It offers a valuable tool for future research into neural computation and learning.