What do Reinforcement Learning Models Measure? Interpreting Model Parameters in Cognition and Neuroscience

Area of Science:

• Cognitive Science
• Neuroscience
• Machine Learning

Background:

• Reinforcement learning (RL) is a key concept across multiple scientific disciplines.
• Discrepancies in RL definitions hinder cross-field interpretation and translation of findings.
• This paper critically examines the application and interpretation of RL models within cognitive (neuro)science.

Purpose of the Study:

• To address the over-interpretation of RL modeling results in cognitive science.
• To highlight implicit assumptions regarding the generalizability of RL findings.
• To question the assumption of unique cognitive processes measured by RL parameters (interpretability).

Main Methods:

• Comparative analysis of RL definitions across disciplines.
• Critical review of empirical evidence regarding RL model generalizability.
• Examination of RL parameter interpretability in cognitive neuroscience studies.

Main Results:

• RL findings often lack generalizability across tasks, models, and populations, contrary to common assumptions.
• RL model parameters frequently capture context-dependent functions rather than distinct, stable cognitive processes.
• Implicit assumptions about generalizability and interpretability lead to over-interpretation of RL results.

Conclusions:

• Future computational research requires greater scrutiny of implicit assumptions in RL modeling.
• A systematic understanding of contextual factors is crucial for accurate RL application.
• Improving RL's explanatory power in brain and behavior necessitates addressing these interpretive challenges.