Neighboring State-Aware Policy for Deep Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Deep reinforcement learning (DRL) methods train agents using sequential actions for task completion.
• Current DRL approaches often exhibit short-sightedness due to reliance on the current state for decision-making.
• Improving policy quality and approaching global optimality remain key challenges in DRL.

Purpose of the Study:

• To enhance existing Deep Reinforcement Learning methods by incorporating a neighboring state sequence.
• To overcome the limitations of current state-dependent decision-making in DRL.
• To improve policy learning and agent performance through a more global perspective.

Main Methods:

• Proposed a neighboring state-aware policy integrating past and future states with the current state.
• Concatenated neighboring states and the current state as input to the actor network for action generation.
• Developed two specific implementations of the proposed approach.

Main Results:

• Demonstrated significant enhancement of ten representative DRL methods across nine diverse tasks.
• Validated the effectiveness of the neighboring state-aware policy using three key metrics, including return.
• Showcased improved policy learning and agent performance compared to baseline DRL methods.

Conclusions:

• The neighboring state-aware policy effectively improves DRL performance by providing a global perspective.
• Incorporating neighboring states mimics human decision-making, leading to better understanding of state evolution.
• The proposed approach offers a promising direction for advancing Deep Reinforcement Learning capabilities.