Related Experiment Video
Updated: Jan 15, 2026
Deep Neural Networks for Image-Based Dietary Assessment
Published on: March 13, 2021
Performance Evaluation of 3 Large Language Models for Nutritional Content Estimation from Food Images
Jonatan Fridolfsson1,2, Emma Sjöberg3, Meri Thiwång3
1Center for Lifestyle Intervention, Institute of Medicine, University of Gothenburg, Gothenburg, Sweden.
Background:
Traditional dietary assessment methods face limitations including recall bias, participant burden, and portion size estimation errors. Recent advances in artificial intelligence, particularly multimodal large language models (LLMs), offer potential solutions for automated nutritional analysis from food images.
Objectives:
This study aims to evaluate and compare the performance of 3 leading LLMs (ChatGPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro) in estimating food weight, energy content, and macronutrient composition from standardized food photographs.
Methods:
We analyzed 52 standardized food photographs including individual food components (n = 16) and complete meals (n = 36) in 3 portion sizes (small, medium, large). Each model received identical prompts to identify food components and estimate nutritional content using visible cutlery and plates as size references. Model estimates were compared against reference values obtained through direct weighing and nutritional database analysis (Dietist NET). Performance metrics included mean absolute percentage error (MAPE), Pearson correlations, and systematic bias analysis using Bland-Altman plots.
Results:
ChatGPT and Claude demonstrated similar accuracy with MAPE values of 36.3% and 37.3% for weight estimation, and 35.8% for energy estimation. Gemini showed substantially higher errors across all nutrients (MAPE 64.2%-109.9%). Correlations between model estimates and reference values ranged from 0.65 to 0.81 for ChatGPT and Claude, compared with 0.58-0.73 for Gemini. All models exhibited systematic underestimation that increased with portion size, with bias slopes ranging from -0.23 to -0.50.
Conclusions:
ChatGPT and Claude achieved accuracy levels comparable with traditional self-reported dietary assessment methods but without associated user burden, suggesting potential utility as dietary monitoring tools. However, systematic underestimation of large portions and high variability in macronutrient estimation indicate these general-purpose LLMs are not yet suitable for precise dietary assessment in clinical or athletic populations where accurate quantification is critical.
