On the Equilibrium Between Feasible Zone and Uncertain Model in Safe Exploration

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Safe exploration is crucial in reinforcement learning (RL) for environmental safety.
• Current methods limit exploration to feasible zones but don't maximize or identify these zones effectively.

Purpose of the Study:

• To address the unresolved questions of maximizing and identifying feasible zones in safe exploration.
• To propose a novel framework for achieving equilibrium between feasible zones and environment models in RL.

Main Methods:

• Introduced the Safe Equilibrium Exploration (SEE) framework, an equilibrium-oriented approach.
• Utilized a graph formulation to represent the uncertain environment model.
• Alternated between expanding the feasible zone and refining the environment model.

Main Results:

• Proved that SEE monotonically refines the uncertain model and expands feasible zones.
• Demonstrated that both the model and feasible zone converge to a safe exploration equilibrium.
• Achieved significant expansion of feasible zones with zero constraint violations in experiments.

Conclusions:

• The interdependence of feasible zones and environment models is key to safe exploration.
• SEE provides the first framework to achieve equilibrium for maximized safe exploration.
• The proposed method is effective in expanding exploration capabilities while maintaining safety guarantees.