Reinforcement Learning and Episodic Memory in Humans and Animals: An Integrative Framework

Area of Science:

• Psychology
• Neuroscience
• Computational Neuroscience

Background:

• Reinforcement learning (RL) has advanced significantly, primarily through studies of simplified tasks.
• Real-world RL involves high-dimensional, continuous, and partially observable states, leading to sparse data.
• Classical RL assumptions are often violated by long-term reward dependencies.

Purpose of the Study:

• To review the psychology and neuroscience of reinforcement learning.
• To propose integrating episodic memory into RL systems to overcome computational challenges.
• To explore how memory contributes to an integrated learning system.

Main Methods:

• Review of computational theories of RL.
• Examination of empirical evidence supporting the role of memory in RL.
• Analysis of how episodic memory addresses sparse data and long-term dependencies.

Main Results:

• Episodic memory can enhance RL by enabling efficient value function approximation in complex state spaces.
• Integrating episodic memory allows RL systems to learn effectively from limited data.
• Episodic memory aids in bridging long-term action-reward dependencies.

Conclusions:

• The computational challenges in real-world reinforcement learning can be mitigated by incorporating episodic memory.
• Episodic memory is crucial for efficient and adaptive learning in complex environments.
• Memory plays a vital, integrated role within reinforcement learning systems.