Related Experiment Video
Updated: Jun 27, 2026

Deep Neural Networks for Image-Based Dietary Assessment
Published on: March 13, 2021
Model architecture dominates nutritional estimation accuracy in vision-language systems
Luca Vedovelli1,2, Sofia Pugnaloni1,3, Corrado Lanera1,2
1Unit of Biostatistics, Epidemiology, and Public Health, Department of Cardiac, Thoracic, Vascular Sciences and Public Health, University of Padua, Padua, Italy.
Abstract:
Vision-language models (VLMs) enable automated dietary assessment from food photographs, yet optimal deployment strategies remain unestablished. We evaluated 40 VLMs across eight providers using Nutrition5k database and original dishes pictures, testing image configurations (1-5 angles), prompt strategies (7 approaches), and image quality conditions, benchmarked against professional nutritionists. Model architecture dominated performance variance (99.6%). Multi-angle photography provided no benefit over single images (RMSLE: 0.627 vs. 0.623, p = 0.182), and prompt engineering showed no significant effects after correction (all p > 0.05). Contrary to conventional assumptions favoring standardized conditions, high-quality consumer smartphone photography significantly outperformed controlled laboratory imaging despite 56% smaller sample size (RMSLE: 0.548 vs. 0.616, p = 0.020, Cliff's δ=-0.58). Ingredient descriptions slightly improved aggregate performance (0.548→0.516, p = 0.039) with heterogeneous model responses (- 19.1% to + 3.4%). Professional nutritionists substantially outperformed all VLMs (RMSLE: 0.176 vs. 0.443 best AI, 152% gap), particularly for protein estimation (672% worse AI performance). These results establish that model selection and image quality dominate nutritional estimation accuracy, while multi-view imaging and prompt complexity are negligible factors. Current VLMs suit consumer calorie tracking applications; achieving clinical-grade macronutrient profiling will require advances in model architecture rather than deployment optimization.