Evaluation of large language models for carbohydrate counting in type 1 diabetes mellitus: a comparative study with...

To evaluate large language models (LLMs) accuracy in carbohydrate (CHO) counting (CC) and compare their performance with estimations by patients with type-1 diabetes mellitus (T1DM).

This cross-sectional study included 14 adults with T1DM with advanced CC training and > 70% accuracy on the AdultCarbQuiz. Eighteen main meals and four snacks were prepared by dietitians using standardized descriptions and two-angle photographs. Dietitian-determined values served as the reference. CHO estimations by patients and LLMs (ChatGPT-5.2, Gemini-3 Pro, and Claude Sonnet-4.5) were compared. LLM performance was evaluated in two-stages: image-only (stage-1) and image-plus-detailed-description (stage-2) inputs. Performance was assessed using mean absolute error (MAE), mean absolute percentage error (MAPE), counting accuracy (CA; ≤10 g), Wilcoxon signed-rank test, and Bland-Altman analysis.

Patients showed MAE 8.31 g, MAPE 13.82%, and CA 73.74%. In Stage-1, Gemini had the lowest MAE (6.55 g), and MAPE (12.54%), while Claude achieved the highest CA (81.8%). In Stage-2, Gemini outperformed patients (p < 0.001) with MAE 2.14 g, MAPE 3.21%, CA 100%, and mean bias - 1.05 g. Other model-stage combinations showed performance comparable to patients.

LLMs, particularly Gemini-3 Pro, may support CC when detailed inputs are provided; however, findings should be interpreted cautiously given the small, highly selected sample.