Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach.

Area of Science:

• Neuroscience
• Computational Neuroscience
• Machine Learning

Background:

• Extinction learning's context-dependence is known to require the hippocampus, but its neural basis remains unclear.
• Existing models often rely on explicit context representations or signals, which may not reflect biological reality.

Purpose of the Study:

• To investigate the neural mechanisms underlying the hippocampus's role in context-dependent extinction learning.
• To model how context-dependence can emerge from raw sensory inputs without explicit context encoding.

Main Methods:

• Developed a memory-driven reinforcement learning model using deep neural networks.
• Trained agents to navigate biologically realistic virtual reality environments using only raw camera inputs.
• Compared the behavior of memory-intact agents with memory-impaired agents.

Main Results:

• Memory-intact agents developed distinct context representations and exhibited ABA renewal (a form of context-dependent behavior).
• Memory-impaired agents failed to show ABA renewal, mimicking hippocampal lesion effects.
• The model demonstrated that context-dependence can emerge solely from visual inputs and internal memory dynamics.

Conclusions:

• The hippocampus's role in context-dependent extinction learning may arise from its function in episodic-like memory, rather than explicit context representation.
• Context-dependence in learning can emerge computationally from raw sensory data and memory processes.
• This work provides a novel computational framework for understanding hippocampal function in associative learning and memory.