J-Mac: Jacobian Matrix Meets Masked Contrastive Learning for Generalization in Reinforcement Learning

Area of Science:

• Artificial Intelligence
• Computer Vision
• Robotics

Background:

• Reinforcement Learning (RL) struggles with generalization in vision-based tasks due to environmental variations.
• Existing methods offer limited improvements for significant domain shifts between training and real-world scenarios.

Purpose of the Study:

• To introduce Jacobian Matrix Meets Masked Contrastive Learning (J-Mac), an auxiliary task for enhancing RL generalization.
• To address the challenge of significant environmental changes in vision-based RL applications.

Main Methods:

• J-Mac learns correlations between visual states using transition dynamic learning.
• It employs masked contrastive learning to remove task-irrelevant features from visual state representations.

Main Results:

• The proposed J-Mac method significantly boosts the generalization capabilities of various base RL algorithms.
• J-Mac outperforms state-of-the-art methods on diverse vision-based benchmarks.

Conclusions:

• J-Mac effectively enhances RL generalization, particularly in scenarios with substantial environmental differences.
• The approach shows promise for robust vision-based RL applications in dynamic environments.