Auto adaptation of closed-loop insulin delivery system using continuous reward functions and incremental...

Area of Science:

• Biomedical Engineering
• Control Systems
• Artificial Intelligence

Background:

• Artificial Pancreas (AP) systems, also known as Automated Insulin Delivery (AID) systems, are crucial for Blood Glucose (BG) regulation.
• Current AID systems often focus on short-term performance, neglecting long-term adaptation to physiological variations like Insulin Sensitivity (IS).
• Inadequate adaptation to IS variations can lead to suboptimal performance in AID systems.

Purpose of the Study:

• To evaluate the long-term adaptation capabilities of two Reinforcement Learning (RL) agents for a Fully Automated Insulin Delivery (fAID) system.
• To compare the performance of RL agents trained with piecewise and continuous reward functions.
• To assess an adaptive discretization scheme for managing state space exploration in RL-based AID systems.

Main Methods:

• In-silico evaluation of two RL agents for the Automatic Regulation of Glucose (ARG) algorithm.
• Implementation of an adaptive discretization scheme to dynamically expand the state space.
• Training RL agents using both piecewise and continuous reward functions to handle variations in Insulin Sensitivity (IS).

Main Results:

• Both RL agents demonstrated improved performance over rule-based and baseline controllers for most of the adult population.
• The RL agents effectively adapted to long-term variations in Insulin Sensitivity (IS).
• A continuous shaped reward function resulted in superior performance compared to a piecewise reward function.

Conclusions:

• RL agents offer a promising approach for long-term adaptation in automated insulin delivery systems.
• Continuous reward functions enhance the efficacy of RL agents in AID systems.
• The proposed adaptive discretization scheme aids in efficient state space management for improved control.