Visual Reinforcement Learning Control With Instance-Reweighted Alignment and Instance-Dimension Uniformity.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Visual reinforcement learning (VRL) excels at learning from complex visual data but suffers from representation degradation due to complete and dimensional collapse.
• Existing contrastive learning methods in VRL can mitigate complete collapse but often face the class collision dilemma, hindering effective learning.

Purpose of the Study:

• To propose a novel VRL control method, instance-reweighted alignment and instance-dimension uniformity (IAIU), designed to address representation degradation and class collision.
• To enhance the robustness and efficacy of VRL by improving state representation learning from high-dimensional visual inputs.

Main Methods:

• IAIU employs instance-reweighted alignment by minimizing Kullback-Leibler (KL) divergence to align state representations within semantic classes, mitigating class collision.
• A regularization mechanism using the Hilbert-Schmidt independence criterion (HSIC) and orthogonal constraints ensures instance-dimension uniformity, suppressing collapse phenomena.
• The method focuses on extracting task-relevant state representations through a dual strategy of alignment and uniformity.

Main Results:

• IAIU demonstrated superior performance on the distracting control suite (DCS) benchmark compared to existing methods.
• The proposed method showed substantial enhancements in both representational ability and policy efficacy.
• Simulation results validate the effectiveness of IAIU in overcoming the limitations of previous VRL approaches.

Conclusions:

• IAIU effectively addresses critical challenges in VRL, including class collision and representation collapse, by integrating alignment and uniformity principles.
• The method ensures the extraction of robust and informative state representations, leading to improved policy performance.
• IAIU represents a significant advancement in VRL, offering enhanced capabilities for learning from complex visual environments.