Physics-informed Koopman-constrained implicitQ-learning for safe offline reinforcement learning in mechanical...

Area of Science:

• Biomedical Engineering
• Artificial Intelligence
• Critical Care Medicine

Background:

• Mechanical ventilation requires precise parameter optimization within safety limits.
• Offline learning presents challenges in utilizing retrospective clinical data for policy optimization.
• Integrating physical laws and ensuring policy robustness are critical for safe AI-driven ventilation.

Purpose of the Study:

• To present a novel offline reinforcement learning framework, PIK-IQL, for mechanical ventilation management.
• To address challenges in learning from retrospective data, incorporating physical constraints, and ensuring policy robustness.
• To develop a Physics-Informed Koopman world model for interpretable, long-horizon predictions of respiratory dynamics.

Main Methods:

• Developed a Physics-Informed Koopman world model to linearize nonlinear respiratory dynamics while preserving mechanical constraints.
• Employed Koopman-constrained Implicit Q-Learning (IQL) with expectile regression for conservative policy learning from offline data.
• Implemented hierarchical action regularization to prevent excessive parameter adjustments and ensure safety.

Main Results:

• PIK-IQL demonstrated a policy value improvement of 0.088 over the behavior policy on the MIMIC-IV database.
• The framework achieved zero aggressive actions exceeding safety thresholds during experiments.
• Rosenbaum sensitivity analysis confirmed high causal robustness (Γ * = 3.0) against unmeasured confounding.

Conclusions:

• PIK-IQL offers a robust and safe framework for optimizing mechanical ventilation using offline data and physics-informed AI.
• The approach effectively integrates respiratory mechanics and data-driven policy optimization, enhancing clinical decision-making.
• The method shows promise for improving patient outcomes in intensive care units through reliable AI-driven interventions.