C2RL: Convolutional-Contrastive Learning for Reinforcement Learning Based on Self-Pretraining for Strong...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Reinforcement learning (RL) agents struggle with generalization in unseen environments, especially with high-dimensional image inputs.
• Self-supervised learning (SSL) with data augmentation aids RL generalization but can be disrupted by excessive image alterations.

Purpose of the Study:

• To propose a novel contrastive learning framework to enhance generalization in reinforcement learning.
• To effectively manage the trade-off between RL performance and data augmentation strength for improved generalization.

Main Methods:

• Developed a contrastive learning approach integrated into an RL architecture.
• Utilized data augmentation strategies within the contrastive learning framework to maximize auxiliary learning effects.
• Investigated the impact of varying data augmentation strengths on RL performance and generalization.

Main Results:

• The proposed contrastive learning method successfully leverages strong data augmentation.
• Demonstrated superior generalization capabilities compared to existing methods on the DeepMind Control suite.
• Showcased that strong augmentation, when managed by contrastive learning, enhances rather than disrupts RL.

Conclusions:

• Contrastive learning offers an effective solution for the RL generalization problem with image inputs.
• The proposed method enables the use of strong data augmentation to boost generalization without compromising RL task performance.
• This framework advances the robustness of reinforcement learning agents in diverse and unseen testing scenarios.