High-Level Decision Making in a Hierarchical Control Framework: Integrating HMDP and MPC for Autonomous Systems

Area of Science:

• Control Systems Engineering
• Artificial Intelligence
• Robotics

Background:

• Autonomous systems face challenges integrating discrete decision-making with continuous dynamics.
• Evolving operational environments necessitate adaptive control strategies.
• Existing models often struggle to bridge high-level discrete logic and low-level continuous control.

Purpose of the Study:

• To propose a comprehensive framework for autonomous decision-making in hybrid systems.
• To develop a novel modeling approach integrating discrete and continuous dynamics.
• To ensure safety and optimality in autonomous control design.

Main Methods:

• Modeled the decision-making system as a hybrid system: a controlled Markov decision process (MDP) coupled with autonomous continuous dynamics, termed hybrid Markov decision process (HMDP).
• Developed a decision-making scheme using model predictive control (MPC) for the HMDP.
• Ensured recursive feasibility and stability through control design and analysis.

Main Results:

• The proposed HMDP framework effectively integrates discrete and continuous state variables for robust decision-making.
• Model predictive control (MPC) implementation guarantees recursive feasibility and stability.
• Simulations demonstrate the framework's capability in handling complex, dynamic environments, exemplified by autonomous lane changing.

Conclusions:

• The developed hybrid framework offers a robust solution for autonomous decision-making challenges.
• The integration of MDPs and continuous dynamics provides enhanced safety and optimality.
• The framework shows significant promise for applications in intelligent transportation systems and beyond.