Deep learning for blood glucose prediction: Reproducibility challenges and factors affecting differential performance

Blood glucose prediction is a critical component of next-generation diabetes technologies, such as artificial pancreas systems, where reliable performance is essential for safety and effectiveness. Although deep learning methods have achieved promising advances in this area, a critical gap remains in understanding the reproducibility and generalizability of these methods. To contextualize the gap, this study reviewed 67 recent papers that proposed a deep learning method for glucose prediction to identify key reproducibility challenges. Next, we adopted a standardized framework, encompassing technical, statistical, and conceptual reproducibility evaluations, to experimentally assess the reproducibility of eight representative deep learning methods. To achieve this, we reimplemented and evaluated these eight deep learning methods using over 1.36 million continuous glucose monitoring samples (5,061 days) from 128 individuals with type 1 diabetes across three public datasets: OhioT1DM, DiaTrend, and T1DEXI. We found that even though these models demonstrated good technical and statistical reproducibility, their conceptual reproducibility-the ability to generalize to datasets with different diabetes management patterns-was limited. Further analyses revealed that each model's overall prediction performance was strongly influenced by individual glycemic control, with higher prediction errors observed among participants with lower time with blood glucose in the target range (70-180 mg/dL). This study identified key reproducibility challenges associated with current blood glucose prediction methods within type 1 diabetes populations, highlighting the need for increased transparency, dataset diversity, standardized evaluation practices, and code accessibility to ensure reproducible and reliable models for blood glucose prediction.