K-cql: an arterial blood gas analysis-based deep offline reinforcement learning algorithm for mechanical ventilation...

Mechanical ventilation is employed as a supportive therapy for patients with respiratory failure, but the optimal ventilator settings for patient are often unknown and rely on manual adjustment by physicians. Improper parameter settings may lead to severe complications such as lung injury. To personalize mechanical ventilation and predict the optimal ventilator parameters for patients, we propose a ventilator parameter tuning algorithm. This algorithm integrates clinical expertise in ventilator tuning via Arterial Blood Gas (ABG) analysis with data-driven methods. We perform K-means clustering algorithm on patient dataset based on ABG values for the first time, and the classified data was used to train a deep offline reinforcement learning model based on conservative Q-learning (CQL), therefore we named it the K-CQL algorithm. The introduction of human expert knowledge improves the effectiveness of the entire model. Our evaluation based on Fitted Q Evaluation (FQE) on the MIMIC-III dataset shows that the expected return of the output strategy of K-CQL is 1.76 times that of the physicians, and more importantly, the introduction of intermediate rewards related to ABG analysis further improves it. We also demonstrated that the algorithm is capable of recommending mechanical ventilation parameters within a safe range according to clinical nursing standards.