Relating Human Error-Based Learning to Modern Deep RL Algorithms.

Area of Science:

• Cognitive Science
• Machine Learning
• Neuroscience

Background:

• Human error-based learning utilizes directed error for action updates.
• Deep reinforcement learning (RL) employs scalar rewards for similar updates.
• The relationship between RL and human learning remains underexplored.

Purpose of the Study:

• To systematically compare major deep RL algorithms with human error-based learning.
• To develop a novel RL algorithm inspired by human learning mechanisms.
• To evaluate the performance and robustness of the new algorithm.

Main Methods:

• Comparative analysis of three major deep RL algorithm families against human error-based learning.
• Utilizing a mirror-reversal perturbation experiment to assess differences.
• Development and testing of a new model-based deterministic policy gradients (MB-DPG) algorithm.

Main Results:

• All three major deep RL approaches showed qualitative differences from human error-based learning.
• MB-DPG successfully captured human error-based learning under mirror-reversal and rotational perturbations.
• MB-DPG demonstrated faster learning on complex tasks and greater robustness to model misspecification compared to standard RL algorithms.

Conclusions:

• Deep RL algorithms are fundamentally different from human error-based learning.
• MB-DPG offers a promising approach to bridge this gap, enhancing RL capabilities.
• The findings suggest new directions for developing more human-like AI learning systems.