Generalized attention-weighted reinforcement learning.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Artificial Intelligence

Background:

• Attention and reinforcement learning (RL) interact in neuroscience to simplify task representations.
• Attention mechanisms are popular in machine learning but underutilized in decision-making problems.

Purpose of the Study:

• To design a deep RL paradigm based on the attention-weighted RL (AWRL) model from computational neuroscience.
• To demonstrate that self-attention combined with value function approximation is a general formulation of AWRL.
• To evaluate the agent's performance on complex Atari tasks with relevant and irrelevant features.

Main Methods:

• Leveraged the theoretical attention-weighted RL (AWRL) model.
• Developed a deep RL agent incorporating self-attention and value function approximation.
• Trained and evaluated the agent on three Atari tasks of varying complexity.

Main Results:

• Agent performance relies on compiling compound features, not just focusing on individual ones.
• Self-attention provides high resiliency to noise (irrelevant features) compared to benchmarks.
• Both bottom-up and top-down attention mechanisms contribute significantly to the learning process.

Conclusions:

• The AWRL framework is broadly valid for complex decision-making scenarios.
• Integrating neuroscience-derived models with RL offers significant benefits for machine learning.
• This approach enhances agent decision-making capabilities in complex environments.