Safe and Balanced: A Framework for Constrained Multi-Objective Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Control Systems

Background:

• Reinforcement learning (RL) in safety-critical systems faces challenges in balancing multiple objectives with strict safety constraints.
• Existing methods struggle with conflicting gradients when optimizing diverse objectives simultaneously.

Purpose of the Study:

• To propose a novel primal-based framework for safe multi-objective reinforcement learning (RL).
• To effectively balance multiple objectives while strictly adhering to safety constraints in RL.
• To address the issue of conflicting gradients in multi-objective RL.

Main Methods:

• A primal-based framework is developed to orchestrate policy optimization between multi-objective learning and constraint adherence.
• A novel natural policy gradient manipulation method is employed to optimize multiple RL objectives.
• The algorithm rectifies policies to minimize constraint violations when they occur.

Main Results:

• The proposed method successfully optimizes multiple RL objectives while ensuring constraint adherence.
• Theoretical convergence and constraint violation guarantees are established.
• Superior performance compared to state-of-the-art methods on challenging safe multi-objective RL tasks was demonstrated.

Conclusions:

• The novel framework provides a robust solution for safe multi-objective reinforcement learning.
• The method effectively handles conflicting gradients and ensures safety in critical applications.
• This approach advances the capabilities of RL in safety-critical domains.