Related Experiment Video
Updated: Mar 29, 2026
Deep Neural Networks for Image-Based Dietary Assessment
Published on: March 13, 2021
Accuracy of AI-Based Nutrient Estimation from Standardized Hospital Meal Images: A Comparison with Registered
Tomomi Isobe1,2, Lim Wan Zhang1, Hana Murakami3
1Graduate School of Comprehensive Scientific Research, Prefectural University of Hiroshima, Hiroshima 734-8558, Japan.
Background:
Accurate dietary assessment is vital for preventing malnutrition in aging populations, particularly in home-care settings. Although Large Multimodal Models (LMMs) for nutrient estimation are evolving, their nutrient-specific accuracy requires rigorous validation.
Methods:
Fifteen standardized hospital meals were photographed under controlled conditions (90-degree angle, 500 lux). Ground truth values were determined by direct weighing. Estimates for energy and macronutrients were performed by 10 registered dietitians (RDs) and 10 AI models (including ChatGPT-4o and Gemini 1.5 Pro). Accuracy was assessed using Pearson's correlation, Mean Absolute Error (MAE), and Bland-Altman analysis to quantify systematic bias.
Results:
For energy and carbohydrates, RDs and top-performing AI models (notably ChatGPT-4o and Gemini 1.5 Pro) demonstrated practical accuracy (r > 0.8, frequently within ±10% range). However, accuracy for protein and lipids was significantly lower across all AI models. Specifically, all AI models exhibited a substantial systematic overestimation of lipids (Mean Bias > +20%, p < 0.01), highlighting a critical "invisible nutrient" bias.
Conclusions:
Current AI tools show potential for caloric and carbohydrate monitoring but struggle with lipid and protein density. These findings emphasize the need for human-AI collaboration ("human-in-the-loop") and the integration of cooking metadata to improve clinical utility in geriatric nutrition.
