Related Experiment Video
Updated: Aug 5, 2026
12:08
Improving IV Insulin Administration in a Community Hospital
Published on: June 11, 2012
INSIGHT: Ingest and Standardize Glucose Harmonization Tool
Ben Ehlert1,2, Krish Shah1, Dev Katarya1
1Department of Genetics, Stanford University School of Medicine, CA, USA.
Journal of Diabetes Science and Technology
|July 28, 2026
Summary
INSIGHT, a novel tool, harmonizes fragmented continuous glucose monitoring (CGM) datasets using large language models (LLMs). This streamlines data for machine learning and research, improving accessibility and enabling pooled analyses.
Area of Science:
- Biomedical Informatics
- Data Science
- Endocrinology
Background:
- Publicly available continuous glucose monitoring (CGM) datasets are fragmented across diverse schemas, hindering data reuse, pooling, and machine learning (ML) development.
- Manual data ingestion and standardization of smaller CGM studies present significant barriers to accessibility and integration.
Purpose of the Study:
- To develop an open-source, LLM-assisted pipeline for harmonizing heterogeneous public CGM datasets into a minimal common schema.
- To reduce the manual effort and cost associated with preparing CGM data for large-scale analysis and ML applications.
Main Methods:
- Developed INSIGHT (Ingest and Standardize Glucose Harmonization Tool), an LLM-assisted harmonization pipeline.
- Combined deterministic rules with LLM assistance for file identification, role classification, schema generation, data normalization (timestamps, units), subject ID resolution, and merging.
- Evaluated INSIGHT on 14 public CGM datasets against curated reference outputs, assessing performance metrics including precision, recall, subject recovery, temporal alignment, and glucose fidelity.
Main Results:
- Top INSIGHT configurations achieved near-perfect harmonization fidelity on held-out data.
- LLMs (GPT-5.4, Gemini 3.1 Pro Preview) demonstrated high performance (mean INSIGHT scores of 0.990) with minimal glucose error (MAE < 0.03 mg/dL).
- Harmonization errors were mainly related to complex subject-timestamp recovery in nested datasets, not glucose unit conversion.
Conclusions:
- LLM-assisted code generation effectively generalizes CGM data harmonization across varied datasets, producing auditable data loaders.
- INSIGHT reduces data reuse costs, complements existing repositories, and facilitates pooled analyses and foundation model development in glycemic physiology.
