A Robust Multi-Virtual-Agent Inverse Reinforcement Learning Approach With Data Aggregation for Perturbed...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Learning control in uncertain environments is a significant challenge.
• Conventional imitation learning (IL) and inverse reinforcement learning (IRL) have limitations in handling perturbations and ensuring policy resilience.
• Existing methods struggle with repeatability and robustness against environmental uncertainties.

Purpose of the Study:

• To propose a novel Multi-Virtual-Agent Inverse Reinforcement Learning (MVIRL) method for generating stable and resilient control policies.
• To address the limitations of current IL and IRL techniques in managing environmental uncertainties and perturbations.
• To develop a method that enhances the robustness and reliability of AI control systems.

Main Methods:

• Designed multiple virtual agents interacting with relevant environments to recover a resilient reward function.
• Incorporated consideration of upper and lower bounds for comprehensive perturbation coverage.
• Utilized maximum discrimination for worst-case scenarios and data aggregation to reduce demonstration requirements.

Main Results:

• The MVIRL method recovered a reward function that effectively handles perturbations by considering bounds.
• The method demonstrated improved ability to handle uncertainties, requiring fewer demonstrations than existing approaches.
• Case studies involving gravity and noise interruptions validated the method's effectiveness.

Conclusions:

• The proposed MVIRL method significantly outperforms comparable IL and IRL methods based on average return and standard deviation metrics.
• MVIRL exhibits superior robustness to varying levels of uncertainty and perturbations.
• The approach offers a more stable and resilient policy for AI control in challenging environments.