Addressing implicit bias in adversarial imitation learning with mutual information.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Adversarial imitation learning (AIL) trains automated decision systems by mimicking expert demonstrations.
• AIL algorithms suffer from implicit bias in their reward functions, leading to sample inefficiency.

Purpose of the Study:

• To address the sample inefficiency caused by reward function bias in AIL.
• To propose Mutual Information Generative Adversarial Imitation Learning (MI-GAIL) for unbiased reward function design.

Main Methods:

• Developed two guidelines for designing unbiased reward functions.
• Shaped the discriminator's reward function by incorporating auxiliary information from a potential-based reward function.
• Utilized mutual information principles to guide reward shaping.

Main Results:

• MI-GAIL demonstrated improved sample efficiency compared to state-of-the-art imitation learning algorithms.
• The proposed method enhanced training stability in continuous control tasks.
• Experimental results validated the effectiveness of MI-GAIL in mitigating AIL reward bias.

Conclusions:

• MI-GAIL effectively corrects implicit bias in AIL reward functions.
• The proposed approach offers a promising direction for improving the performance of imitation learning systems.
• This work provides a foundation for developing more sample-efficient and stable automated decision systems.