Guideline-informed reinforcement learning for mechanical ventilation in critical care

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Decision Support Systems
• Computational Health

Background:

• Reinforcement Learning (RL) shows promise in healthcare for clinical decision-making using observational data.
• Integrating existing medical knowledge into RL is crucial for safety, balanced outcomes, and clinician trust.
• Current RL applications in healthcare face challenges in incorporating established medical guidelines.

Purpose of the Study:

• To present a novel framework for integrating medical guideline knowledge into Reinforcement Learning (RL) algorithms.
• To enhance RL solutions by enforcing safety constraints and optimizing the balance between short- and long-term patient outcomes.
• To facilitate the adoption of RL in clinical practice by aligning AI-driven decisions with established medical expertise.

Main Methods:

• Developed a framework to incorporate knowledge from medical guidelines into RL.
• Implemented components for enforcing safety constraints within the RL framework.
• Introduced an approach to modify the RL learning signal, guided by medical guidelines, for improved outcome balancing.
• Evaluated the framework by enhancing an RL-based mechanical ventilation (MV) system with established ventilation guidelines.

Main Results:

• Off-policy policy evaluation demonstrated the framework's potential.
• The enhanced RL approach showed promise in decreasing 90-day mortality.
• The method successfully ensured lung-protective ventilation strategies were maintained.
• Results indicate a significant improvement in balancing patient outcomes and adhering to clinical guidelines.

Conclusions:

• The presented framework is a significant advancement for implementing RL in clinical settings.
• Integrating medical guidelines into RL enhances safety and optimizes patient outcomes.
• This work paves the way for broader applications of RL in healthcare and suggests future research directions.