Constrained Visual Representation Learning with Bisimulation Metrics for Safe Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Robotics
• Machine Learning

Background:

• Safe reinforcement learning (RL) is crucial for real-world applications, especially those using visual inputs.
• Extracting essential features for safe decision-making from high-dimensional visual data while maintaining sample efficiency remains a challenge.

Purpose of the Study:

• To propose a novel method, Constrained Visual Representation Learning with Bisimulation Metrics (CVRL-BM), for effective and safe reinforcement learning from visual observations.
• To develop a model that learns compact, informative state representations while adhering to safety constraints.

Main Methods:

• CVRL-BM utilizes a sequential conditional variational inference model to compress visual observations into low-dimensional state representations.
• Safety bisimulation metrics are incorporated to quantify state behavioral similarity, guiding the learning of latent state representations.

Main Results:

• Experiments on Safety Gym demonstrate CVRL-BM's superior performance in safety and efficacy compared to existing vision-based safe RL methods.
• CVRL-BM achieved a 19.7% higher reward return and a 41.7% lower cost return than the state-of-the-art Safe SLAC method.
• A 5.0% decrease in cost regret was observed, highlighting the method's effectiveness in minimizing risks.

Conclusions:

• CVRL-BM effectively learns compact and informative visual state representations for safe reinforcement learning.
• The integration of representation learning and safety bisimulation metrics enables robust performance in safety-critical applications.