Data-Driven Dynamic Multiobjective Optimal Control: An Aspiration-Satisfying Reinforcement Learning Approach.

Area of Science:

• Control Theory
• Optimization
• Machine Learning

Background:

• Dynamic multiobjective optimal control problems (MOOCP) are crucial for complex systems.
• Existing methods often require complete system dynamics knowledge, limiting practical application.
• Satisficing decision-making offers a framework for achieving 'good enough' solutions.

Purpose of the Study:

• To develop an iterative, data-driven algorithm for solving dynamic MOOCP in nonlinear continuous-time systems.
• To find Pareto optimal solutions within a dynamic constrained multiobjective framework.
• To enable real-time control using limited system data via a reinforcement learning approach.

Main Methods:

• Leveraging Hamiltonian functionals and inequalities to satisfy objective aspirations.
• Solving relaxed Hamilton-Jacobi-Bellman (HJB) inequalities.
• Developing a sum-of-square (SOS)-based iterative algorithm for aspiration-satisfying MO optimization.
• Proposing a data-driven satisficing reinforcement learning approach for real-time problem-solving.

Main Results:

• The algorithm effectively compares admissible policies using Hamiltonian functionals.
• Hamiltonian inequalities guarantee the satisfaction of objective aspirations.
• The SOS-based algorithm solves the aspiration-satisfying MO optimization problem.
• The data-driven reinforcement learning approach solves the SOS optimization in real-time without full system knowledge.
• Simulation examples validate the proposed algorithm's analytical results.

Conclusions:

• The proposed iterative, data-driven algorithm provides a novel solution for dynamic multiobjective optimal control.
• The method successfully finds Pareto optimal solutions and relates to satisficing decision-making.
• The reinforcement learning approach allows for real-time implementation using trajectory data, overcoming the need for complete system dynamics.