Prescribed Safety Performance Imitation Learning From a Single Expert Dataset.

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Existing safe imitation learning (safe IL) methods primarily focus on replicating expert policies, which may not satisfy diverse or novel safety constraints.
• This limitation hinders the application of safe IL in scenarios requiring adaptable safety specifications.

Purpose of the Study:

• To propose a novel algorithm, Lagrangian Generative Adversarial Imitation Learning (LGAIL), capable of adaptively learning safe policies from expert data under various safety constraints.
• To address the limitations of current safe IL methods in handling diverse safety requirements.

Main Methods:

• Augmenting Generative Adversarial Imitation Learning (GAIL) with safety constraints and relaxing it into an unconstrained problem using a Lagrange multiplier.
• Employing a two-stage optimization: discriminator training for data similarity and forward reinforcement learning for imitation and safety, guided by the dynamically adjusted Lagrange multiplier.

Main Results:

• Theoretical analysis demonstrating the convergence and safety guarantees of LGAIL under prescribed constraints.
• Extensive experiments conducted in OpenAI Safety Gym validating the effectiveness of the proposed approach.

Conclusions:

• LGAIL effectively learns adaptive safe policies that satisfy diverse safety constraints, outperforming existing safe IL methods.
• The dynamic adjustment of the Lagrange multiplier is crucial for balancing imitation performance and safety adherence.