Reinforcement learning for treatment decision-making in sepsis: a scoping review

This scoping review summarizes the progress of reinforcement learning (RL) in clinical decision-making for sepsis at the intersection of medicine and artificial intelligence (AI). All 72 included studies were retrospective, with the majority using the Medical Information Mart for Intensive Care (MIMIC) database (58 studies [80.6%]), and relatively few employing private datasets (10 studies [13.9%]). Study designs varied widely, especially in state representation, action space, reward definition, and choice of algorithms. Most research focused on vasopressor and intravenous fluid management, while fewer studies addressed antibiotics, corticosteroids, mechanical ventilation, heparin, or vasopressin. Although many studies reported RL-derived policies that outperformed clinicians, the reliability and validity of the evaluation methods remain uncertain. Future research should emphasize clinically guided design of states, actions, and rewards, develop rigorous and widely accepted evaluation tools, and explore a broader range of sepsis treatment strategies. Ultimately, advancing interpretability, generalizability, and safety will be critical to effectively integrating RL into routine clinical practice.