Reinforcement learning with Marr.

Area of Science:

• Computational neuroscience
• Cognitive science
• Artificial intelligence

Background:

• Reinforcement learning (RL) is frequently cited as a prime example of David Marr's three levels of scientific inquiry.
• Its computational theories of reward optimization offer algorithmic solutions that resemble neural signals, implying direct neural implementation.

Purpose of the Study:

• To review outstanding questions at each of Marr's three levels of analysis for reinforcement learning.
• To propose a novel approach for advancing understanding in the field.

Main Methods:

• Literature review and theoretical analysis of reinforcement learning.
• Examination of the relationship between computational theories and neural implementations.

Main Results:

• Significant open questions persist at the computational, algorithmic, and neural implementation levels of RL.
• The direct mapping from RL theory to neural signals is not as straightforward as commonly assumed.

Conclusions:

• Progress in understanding reinforcement learning requires integrated insights across all three levels of analysis.
• Future research should emphasize cross-level inspiration rather than solely focusing on mutual constraints between levels.