Deep reinforcement learning for multi-targets propofol dosing

The administration of propofol for sedation or general anesthesia presents challenges due to the complex relationship between patient factors and real-time physiological responses. This study explores the application of deep reinforcement learning (DRL) to automate propofol dosing, aiming to maintain multiple physiological parameters including bispectral index (BIS), heart rate (HR), respiratory rate (RR), and mean arterial pressure (MAP) within safe and desired ranges. A multi-variable pharmacokinetic-pharmacodynamic (PK/PD) simulation environment was developed to model the effects of propofol on the physiological parameters. An adjustable reward system was designed for multi-target anesthetic infusion. The DRL agent was trained using Twin Delayed Deep Deterministic Policy Gradient (TD3), interacting with the simulation environment and receiving rewards for maintaining physiological parameters close to their targets and above safety thresholds. The performance of the TD3 agent was compared to other DRL algorithms and traditional control methods. The TD3 algorithm demonstrated superior performance in achieving precise and safe control of multiple physiological parameters during propofol administration, outperforming other DRL algorithms and traditional control methods. The application of DRL, particularly TD3, offers a promising approach for automating propofol dosing, ensuring better management of physiological parameters and enhancing the safety and effectiveness of sedation and general anesthesia.