Optimizing Loop Diuretic Treatment for Mortality Reduction in Patients With Acute Dyspnea Using a Practical Offline...

Area of Science:

• Clinical informatics
• Machine learning in healthcare
• Reinforcement learning applications

Background:

• Offline reinforcement learning (RL) is increasingly used for clinical decision-making.
• Existing methods lack standardization, leading to overfitted policies and inaccurate evaluations.
• A robust pipeline is needed to improve reliability and clinical workflow integration.

Purpose of the Study:

• Introduce a practical pipeline, Pipeline for Learning Robust Policies in Reinforcement Learning (PROP-RL), for offline RL in clinical settings.
• Enhance policy robustness and minimize disruption to clinical workflows.
• Demonstrate efficacy in optimizing loop diuretic treatment for hospitalized patients.

Main Methods:

• Modeled loop diuretic management as an offline RL problem using electronic health record data.
• Defined a discrete state space, binary action space (diuretic use), and in-hospital mortality reward function.
• Trained and evaluated the policy on a large cohort of adult inpatients, comparing against clinician behavior.

Main Results:

• The study included 36,570 hospitalizations.
• The learned policy aligned with clinicians in most states but diverged in 2 states.
• In cases of divergence, the RL policy significantly reduced mortality from 3.8% to 2.2% (1.6% reduction, P=.006).

Conclusions:

• The PROP-RL pipeline offers a robust approach for offline RL in sequential treatment selection.
• Highlights the importance of state representation and policy evaluation in clinical RL.
• Identifies potential improvements in current loop diuretic treatment strategies and provides a blueprint for future clinical RL applications.