Comparing Deterministic and Stochastic Reinforcement Learning for Glucose Regulation in Type 1 Diabetes

Area of Science:

• Biomedical Engineering
• Artificial Intelligence in Medicine
• Endocrinology

Background:

• Type 1 Diabetes (T1D) necessitates continuous glucose monitoring and insulin therapy.
• Current Artificial Pancreas Systems (APS) require manual input, posing a cognitive burden.
• Reinforcement Learning (RL) offers a potential solution for autonomous glucose regulation in APS.

Purpose of the Study:

• To compare the efficacy of stochastic (PPO) and deterministic (TD3) RL algorithms for glucose regulation in silico.
• To evaluate RL algorithm performance using quantitative, qualitative, and patient-specific metrics.
• To assess the safety and suitability of different RL approaches for APS.

Main Methods:

• In silico evaluation using the UVA/PADOVA 2008 T1D simulator.
• Comparison of Proximal Policy Optimization (PPO) and Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithms.
• Utilized quantitative metrics, qualitative assessments, and patient-specific clinical data.

Main Results:

• Both PPO and TD3 demonstrated potential for autonomous glucose regulation.
• PPO generally showed superior performance compared to TD3 in the simulations.
• TD3 exhibited more interpretable behavior, but this did not consistently correlate with better outcomes.

Conclusions:

• RL algorithms hold promise for reducing the burden of T1D management via APS.
• Further research is needed to enhance the interpretability and predictive performance of RL algorithms in APS.
• Algorithm selection requires careful consideration of both performance and safety/interpretability.