A Novel Model for Arbitration Between Planning and Habitual Control Systems

Area of Science:

• Robotics
• Artificial Intelligence
• Computational Neuroscience

Background:

• Human decision-making involves both habitual action selection and deliberate planning systems.
• Habitual control is efficient but inflexible; deliberate planning is flexible but computationally expensive.
• Integrating these systems offers potential for enhanced robotic control.

Purpose of the Study:

• To propose a general architecture that integrates habitual and deliberate control paradigms.
• To implement and evaluate this architecture in a robotic target-reaching task.
• To demonstrate the model's ability to adapt to changing environments and visual occlusion.

Main Methods:

• A novel architecture with an arbitrator controlling subsystem selection was implemented.
• The system utilized a supervised internal model and deep reinforcement learning for a simulated robotic arm.
• Performance was evaluated under varying target-reaching conditions, including environmental changes and visual occlusion.

Main Results:

• The proposed model rapidly learned system kinematics without prior knowledge.
• It demonstrated robustness to changing environmental rewards, kinematics, and visual occlusion.
• The arbitrator model outperformed exclusive deliberate planning and exclusive habitual control in dynamic and occluded conditions.

Conclusions:

• The integrated architecture effectively harnesses the benefits of both habitual and deliberate control.
• The model exhibits fast learning and adaptive performance in complex, changing environments.
• Internal models are crucial for maintaining performance under visual occlusion, a limitation for pure habitual systems.