Noise Resilience of Successor and Predecessor Feature Algorithms in One- and Two-Dimensional Environments.

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Machine Learning

Background:

• Noisy inputs present significant challenges for reinforcement learning (RL) agents in real-world applications.
• Animals exhibit robust spatial learning in dynamic environments, yet underlying mechanisms are understudied in RL.
• Understanding noise resilience is crucial for developing practical AI systems.

Purpose of the Study:

• To comparatively analyze predecessor feature (PF) and successor feature (SF) algorithms under controlled noise.
• To investigate the impact of noise on RL agent performance in 1D and 2D environments.
• To identify mechanisms for enhancing noise resistance in artificial learning systems.

Main Methods:

• Comparative analysis of SF and PF algorithms against traditional Q-learning.
• Controlled introduction of noise (σ) in 1D and 2D navigation environments.
• Evaluation of cumulative rewards and performance metrics under varying noise levels.

Main Results:

• SF algorithms significantly outperform Q-learning in noise resilience, achieving high cumulative rewards (2216.88±3.83 at σ=0.5 in 1D).
• A nonlinear relationship between noise level and performance was observed in 2D environments, with SF optimal at moderate noise (σ=0.25).
• The λ parameter in PF learning impacts performance, with λ=0.7 showing consistent advantages.

Conclusions:

• SF algorithms offer superior noise resilience for reinforcement learning agents.
• Findings provide insights for developing robust AI in uncertain environments like robotics and autonomous navigation.
• This work bridges computational neuroscience and RL, advancing noise-resistant learning systems.