Feasible Policy Iteration With Guaranteed Safe Exploration.

Area of Science:

• Robotics and Artificial Intelligence
• Machine Learning and Control Systems

Background:

• Safe reinforcement learning (RL) is critical for real-world applications to prevent damage and risks.
• Existing methods often compromise safety during training or address it post-optimality.

Purpose of the Study:

• To propose a feasible policy iteration framework for absolute safety during online RL exploration.
• To ensure constraint violations never occur in real-world interactions.

Main Methods:

• Confining environmental exploration within a dynamically defined feasible region at each step.
• Utilizing a novel constraint decay function with uncertainty for forward invariance.
• Developing practical algorithms with an actor-critic-scenery architecture (safe exploration, model error estimation, network update).

Main Results:

• Achieved performance comparable to baselines with zero constraint violations during training.
• Demonstrated monotonic expansion of the feasible region and policy improvement.
• Contrasted with baseline algorithms requiring numerous violations for similar performance.

Conclusions:

• The proposed framework guarantees absolute safety in online RL exploration.
• Feasible policy iteration shows significant potential for safe deployment in complex real-world systems.
• Enables online evolution of intricate systems without compromising safety.