Hybrid neural-cognitive models reveal how memory shapes human reward learning

Area of Science:

• Cognitive Psychology
• Neuroscience
• Computational Modeling

Background:

• Understanding how past experiences shape future behavior is a key challenge in psychology and neuroscience.
• Reward-guided learning is often explained using reinforcement learning (RL) algorithms, which rely on incrementally updated internal variables.
• Existing RL models may oversimplify the complex mechanisms of human reward learning.

Purpose of the Study:

• To investigate the assumptions of popular reinforcement learning models in human reward learning.
• To develop and test a hybrid modeling approach integrating artificial neural networks and cognitive architectures.
• To determine the necessary and sufficient components for accurate modeling of human reward-guided behavior.

Main Methods:

• A hybrid modeling approach combining artificial neural networks with interpretable cognitive architectures was employed.
• Algorithmic components were estimated in a maximally general form and their necessity/sufficiency systematically evaluated.
• The approach was applied to a large dataset of human reward-learning behavior.

Main Results:

• Successful models necessitate independent and flexible memory variables capable of tracking rich representations of past experiences.
• The findings indicate that human reward learning is not solely based on incremental updating of scalar reward predictions.
• A class of popular RL models based on incremental updating was called into question.

Conclusions:

• Human reward learning relies on more complex memory mechanisms than previously assumed by many RL models.
• Hybrid modeling approaches integrating ANNs and cognitive architectures offer a powerful tool for understanding learning.
• Future research should consider flexible and rich memory representations when modeling reward-guided behavior.