Selective particle attention: Rapidly and flexibly selecting features for deep reinforcement learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Deep Reinforcement Learning (RL) faces challenges with data inefficiency and inflexibility due to end-to-end learning.
• Current methods using pre-existing representations offer partial solutions but remain susceptible to inefficiencies.
• Biological systems utilize selective attention to optimize feature selection for decision-making.

Purpose of the Study:

• To introduce a novel algorithm, Selective Particle Attention (SPA), inspired by biological selective attention.
• To address data inefficiency and inflexibility in Deep RL by enabling selection of feature subsets.
• To develop a method that rapidly and flexibly identifies key features using only reward feedback.

Main Methods:

• Developed Selective Particle Attention (SPA), a novel algorithm for Deep RL.
• SPA selects subsets of existing representations, bypassing slow backpropagation.
• Utilized a particle filter for rapid and flexible feature subset identification based on reward feedback.

Main Results:

• SPA significantly enhances the data efficiency of downstream Deep RL algorithms.
• SPA demonstrates increased flexibility in Deep RL when faced with dynamic task structure changes.
• Evaluated SPA on tasks with raw pixel input and evolving task parameters.

Conclusions:

• SPA offers a biologically inspired solution to improve Deep RL efficiency and flexibility.
• The particle filter approach in SPA provides rapid and adaptable feature selection.
• SPA represents a significant advancement in making Deep RL more robust and applicable to complex, changing environments.