Hierarchical Deep Reinforcement Learning-Based Propofol Infusion Assistant Framework in Anesthesia.

Area of Science:

• Anesthesiology
• Artificial Intelligence
• Machine Learning

Background:

• Automated drug infusion systems are crucial for maintaining stable patient sedation during anesthesia.
• Current methods may lack the adaptability to dynamically adjust drug dosage based on real-time patient physiological data.

Purpose of the Study:

• To develop and evaluate a hierarchical reinforcement learning (RL)-based solution for automated drug infusion.
• To improve the stability of patient sedation using propofol and remifentanil during total intravenous anesthesia (TIVA).

Main Methods:

• The proposed deep infusion assistant policy gradient (DIAPG) model utilizes adversarial autoencoders (AAEs) to learn latent representations of hypnotic depth trajectories.
• A hierarchical RL approach divides the learning policy into a trajectory generative model and a planning policy model.
• The planning policy infers propofol dosage for stable sedation based on generated trajectories.

Main Results:

• The DIAPG model effectively stabilized the bispectral index (BIS) and effect site concentration, even with time-varying target sequences.
• DIAPG demonstrated a 530% performance increase compared to human experts.
• DIAPG showed a 15% performance improvement over standard reinforcement learning algorithms.

Conclusions:

• The DIAPG model offers a robust and effective solution for automated drug infusion in TIVA.
• Hierarchical RL, inspired by AAEs, provides a powerful framework for complex control tasks in anesthesia.
• This AI-driven approach significantly enhances sedation stability and outperforms existing methods.