Safety-Guaranteed Lung-Protective Mechanical Ventilation using Digital Twins and Reinforcement Learning

We present a novel control strategy for automated and individualized mechanical ventilation based on a Reinforcement Learning (RL) controller for a Constrained Markov Decision Process (CMDP). This approach enhances an off-policy actor-critic algorithm known as Primal-Dual Soft Actor-Critic (SAC) with the addition of dual variables to bound critical parameters within clinically acceptable ranges. The proposed technique first creates a high-fidelity computational model (digital twin) of the patient, so that physiological reactions to changes in ventilator settings can be generated in silico to train the RL controller quickly and safely. Once brought online, the novel primal-dual SAC then safely converges to ventilator settings that minimize the risk of ventilator-induced lung injury (VILI), while guaranteeing that the patient's physiological parameters remain within specified safety limits. Correct and robust functioning of the proposed controller is demonstrated by detailed simulations based on real patient data.Clinical relevance- Closed-loop ventilation has been demonstrated to reduce clinician workload, improve patient-ventilator synchrony, and shorten weaning duration in certain settings. However, most existing closed-loop ventilation modes focus solely on achieving oxygenation targets, offering no assurance of minimizing VILI indices or maintaining them within safe limits. In this study, we address this limitation by introducing a safety-guaranteed closed-loop ventilation technique.