Reinforcement Learning Model With Dynamic State Space Tested on Target Search Tasks for Monkeys: Extension to...

Area of Science:

• Computational Neuroscience
• Reinforcement Learning
• Cognitive Modeling

Background:

• Biological systems require adaptive learning to navigate complex and changing environments.
• Episode-dependent learning is crucial for adapting to dynamic situations with varying states.
• Previous models using dynamic state spaces showed promise but required extension for complex tasks.

Purpose of the Study:

• To develop an extended model architecture for episode-dependent learning in complex environments.
• To address limitations in previous models for tasks involving events like initial fixation.
• To improve the adaptability of reinforcement learning agents.

Main Methods:

• Development of a "history-in-episode" architecture, dividing states into episodes and histories.
• Agent actions are selected based on histories within each episode.
• Comparison of the proposed model with the conventional SARSA method in a two-target search task.

Main Results:

• The proposed model achieved near-optimal performance in the two-target search task.
• The model successfully learned to adapt to target-pair switches.
• The conventional SARSA method failed to adapt to target-pair switches, requiring constant relearning.

Conclusions:

• The "history-in-episode" architecture combined with a dynamic state space enables effective episode-dependent learning.
• This model provides a foundation for creating highly adaptable learning systems.
• The proposed approach significantly outperforms conventional methods in complex, dynamic tasks.