Personalized tuning of a reinforcement learning control algorithm for glucose regulation

Area of Science:

• Biomedical Engineering
• Artificial Intelligence in Medicine
• Endocrinology

Background:

• Artificial pancreas systems aim for automated insulin delivery in type 1 diabetes.
• Patient variability necessitates personalized control strategies for optimal glucose regulation.
• Existing systems often require lengthy calibration periods.

Purpose of the Study:

• To develop and evaluate a patient-specific, adaptive control strategy for artificial pancreas systems.
• To implement a reinforcement learning approach, specifically Actor-Critic (AC), for glucose regulation.
• To introduce a novel, personalized initialization method using transfer entropy (TE).

Main Methods:

• An adaptive, patient-specific control strategy based on Actor-Critic (AC) reinforcement learning was developed.
• A personalized initialization method using transfer entropy (TE) to estimate relationships between insulin and glucose signals was designed.
• The AC algorithm, with TE-based, random, and zero initializations, was evaluated in silico across adult, adolescent, and child populations.

Main Results:

• TE-based initialization resulted in faster learning, achieving 98% (adults), 90% (adolescents), and 73% (children) in A+B zones of Control Variability Grid Analysis within five days.
• TE-based initialization outperformed random (95%, 78%, 41%) and zero (93%, 88%, 41%) initializations.
• Children experienced a faster reduction in the daily Low Blood Glucose Index with TE-based tuning.

Conclusions:

• Automatic, personalized tuning using TE significantly reduces the learning period for artificial pancreas systems.
• The proposed TE-based initialization enhances the overall performance and efficiency of the AC algorithm for glucose regulation.
• This approach offers a promising avenue for improving automated insulin delivery and diabetes management.