End-to-end offline reinforcement learning for glycemia control.

Area of Science:

• Biomedical Engineering
• Artificial Intelligence in Medicine
• Endocrinology

Background:

• Closed-loop systems for type I diabetes management heavily depend on simulated patient data.
• Over-fitting simulators can lead to poor performance and safety risks, especially with atypical patient data.
• Current simulation-based approaches may not adequately capture the full spectrum of patient variability.

Purpose of the Study:

• To develop a safer and more adaptable glucose control system for type I diabetes.
• To mitigate the risks associated with over-fitting patient simulators.
• To enable robust glycemia control using real-world patient data.

Main Methods:

• Utilizing model-free offline reinforcement learning (RL) agents trained directly on real patient data.
• Implementing an end-to-end personalization pipeline for agent adaptation.
• Employing offline-policy evaluation methods to eliminate the need for simulators.

Main Results:

• Demonstrated the feasibility of using offline RL agents for glycemia control.
• Showcased an approach that bypasses the need for patient simulators.
• Enabled estimation of clinically relevant diabetes metrics without simulation.

Conclusions:

• Offline RL agents trained on real data offer a promising alternative to simulation-based approaches for type I diabetes glucose control.
• The proposed personalization pipeline enhances adaptability and safety by reducing reliance on potentially over-fitted simulators.
• This method facilitates robust glycemia management and metric evaluation in real-world clinical settings.