Reinforcement learning on slow features of high-dimensional input streams.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Animal Behavior

Background:

• Complex behaviors are learned from multimodal sensory data.
• Reward-based learning algorithms struggle with high-dimensional data.
• The brain's mechanism for learning from high-dimensional input remains an open question.

Purpose of the Study:

• To propose a biologically plausible, two-stage learning system for high-dimensional data.
• To investigate the efficiency of this system in reinforcement learning tasks.
• To support the role of slowness learning in efficient state representation.

Main Methods:

• Developed a hierarchical slow feature analysis (SFA) network for data preprocessing.
• Integrated SFA with a reward-based neural network for learning.
• Validated the system using computer simulations on high-dimensional visual input streams.

Main Results:

• The proposed architecture successfully learned demanding reinforcement learning tasks.
• Learning speed was comparable to using explicit low-dimensional state representations.
• Performance in a Morris water maze-like task matched experimental findings in rats.

Conclusions:

• The generic two-stage learning system effectively handles high-dimensional input for behavioral learning.
• Slowness learning is a crucial unsupervised principle for forming efficient neural state representations.
• This model provides insights into how the brain processes complex sensory information for learning.