Reward-predictive representations generalize across tasks in reinforcement learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Reinforcement learning (RL) with deep neural networks excels in specific tasks but lacks generalization.
• Current transfer learning methods in RL are limited, failing to achieve human-like deep transfer.
• Deep transfer requires discovering abstractions for analogical reuse of representations across distinct tasks.

Purpose of the Study:

• To develop a novel abstraction method for reinforcement learning that enables generalization across diverse tasks.
• To investigate if reward-predictive representations can facilitate deep transfer in artificial agents.
• To explore the potential for state representation abstractions that support equivalence relations.

Main Methods:

• Developed abstractions that minimize prediction errors for reward outcomes.
• Constructed reward-predictive representations that compress task state spaces.
• Combined states based on equivalence in transition and reward functions.

Main Results:

• Reward-predictive abstractions generalized across tasks with varying transition and reward functions.
• These representations are not tied to specific task dynamics, enabling broad applicability.
• The approach contrasts with myopic reward-maximization strategies in RL.

Conclusions:

• Abstractions minimizing reward prediction error facilitate generalization in reinforcement learning.
• This approach offers a pathway towards achieving deep transfer in artificial agents.
• Findings suggest investigating neural and cognitive systems for similar abstraction mechanisms.