Visual Pretraining via Contrastive Predictive Model for Pixel-Based Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Reinforcement learning (RL) often relies on reward-driven learning, which has limitations.
• Vision-based RL requires effective representation learning from visual inputs.

Purpose of the Study:

• To develop an unsupervised learning framework for representation learning in vision-based RL, detached from policy learning.
• To improve the performance and generalization of RL agents using enhanced visual representations.

Main Methods:

• Proposed visual pretraining via contrastive predictive model (VPCPM) framework.
• Utilized forward and inverse dynamics models with contrastive loss for representation learning.
• Initialized convolutional encoders with VPCPM for state-of-the-art RL algorithms.

Main Results:

• Significant performance boost for RAD (44%) and DrQ (10%) at 100 steps.
• VPCPM matches or outperforms existing unsupervised representation learning methods.
• Demonstrated successful generalization of learned representations to new, similar tasks.

Conclusions:

• VPCPM effectively learns representations detached from policy learning, overcoming limitations of reward-driven methods.
• The proposed method enhances performance and generalization in vision-based RL tasks.
• VPCPM offers a robust approach for unsupervised visual representation learning in RL.