Value Decomposition-Based Multi-Agent Learning for Anesthetics Collaborative Control.

Area of Science:

• Artificial Intelligence in Medicine
• Anesthesiology
• Pharmacokinetics and Pharmacodynamics

Background:

• Automated control of personalized multiple anesthetics in Total Intravenous Anesthesia (TIVA) is complex.
• Existing systems like target-controlled infusion (TCI) and closed-loop systems have limitations in personalization and collaborative control.
• These limitations stem from static models and single-anesthetic focus.

Purpose of the Study:

• To introduce a novel Value Decomposition MultiAgent Deep Reinforcement Learning (VD-MADRL) framework for Personalized Multiple Anesthetics Control in a Closed-Loop system (PMAC-CL).
• To optimize the collaborative control between propofol and remifentanil using a Markov Game (MG).
• To enhance credit allocation and collaborative control through value function decomposition methods.

Main Methods:

• Developed a VD-MADRL framework based on a Markov Game (MG).
• Utilized multivariate environment modeling with random forest (RF) for anesthesia state simulation.
• Implemented data resampling and alignment techniques for data validity and Markov property adherence.

Main Results:

• The VD-MADRL framework demonstrated more refined dose adjustments and stable maintenance of anesthesia state indicators compared to human experience.
• The VDN algorithm, a component of VD-MADRL, achieved a 16.4% increase in cumulative reward (CR) and a 58.0% reduction in mean MDPE in general surgery.
• Experiments on general and thoracic surgery datasets validated the framework's efficacy.

Conclusions:

• The proposed VD-MADRL framework offers significant clinical value for personalized multiple anesthetic control in TIVA.
• This AI-driven approach improves the precision and stability of anesthesia management.
• The findings suggest a promising direction for future advancements in automated clinical anesthesia.