Adversarial Imitation Learning With General Function Approximation: Theoretical Analysis and Practical Algorithms.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Reinforcement Learning

Background:

• Adversarial Imitation Learning (AIL) shows practical success using neural networks.
• Theoretical analysis of AIL is limited to simplified settings, creating a theory-practice gap.
• Existing AIL methods often involve complex designs hindering practical application.

Purpose of the Study:

• To bridge the gap between theory and practice in Adversarial Imitation Learning.
• To develop theoretically grounded online AIL methods for general function approximation.
• To introduce a novel framework, optimization-based AIL (OPT-AIL), for efficient policy learning.

Main Methods:

• Introduced optimization-based AIL (OPT-AIL) framework.
• Developed model-free and model-based OPT-AIL variants.
• Utilized online optimization for reward learning and optimism-regularized optimization for policy learning.

Main Results:

• Both OPT-AIL variants achieve polynomial expert sample and interaction complexity.
• Demonstrated provably efficient Adversarial Imitation Learning under general function approximation.
• Empirical results show OPT-AIL outperforms state-of-the-art deep AIL methods.

Conclusions:

• OPT-AIL provides the first provably efficient Adversarial Imitation Learning methods for general function approximation.
• The framework simplifies practical implementation by requiring approximate optimization of two objectives.
• OPT-AIL significantly advances the field by unifying theoretical guarantees with practical applicability.