Related Experiment Video
Updated: Jul 11, 2025

Deep Neural Networks for Image-Based Dietary Assessment
Published on: March 13, 2021
An AI Dietitian for Type 2 Diabetes Mellitus Management Based on Large Language and Image Recognition Models:
Haonan Sun1, Kai Zhang1, Wei Lan2
1School of Life Science, Beijing University of Chinese Medicine, Beijing, China.
Background:
Nutritional management for patients with diabetes in China is a significant challenge due to the low supply of registered clinical dietitians. To address this, an artificial intelligence (AI)-based nutritionist program that uses advanced language and image recognition models was created. This program can identify ingredients from images of a patient's meal and offer nutritional guidance and dietary recommendations.
Objective:
The primary objective of this study is to evaluate the competence of the models that support this program.
Methods:
The potential of an AI nutritionist program for patients with type 2 diabetes mellitus (T2DM) was evaluated through a multistep process. First, a survey was conducted among patients with T2DM and endocrinologists to identify knowledge gaps in dietary practices. ChatGPT and GPT 4.0 were then tested through the Chinese Registered Dietitian Examination to assess their proficiency in providing evidence-based dietary advice. ChatGPT's responses to common questions about medical nutrition therapy were compared with expert responses by professional dietitians to evaluate its proficiency. The model's food recommendations were scrutinized for consistency with expert advice. A deep learning-based image recognition model was developed for food identification at the ingredient level, and its performance was compared with existing models. Finally, a user-friendly app was developed, integrating the capabilities of language and image recognition models to potentially improve care for patients with T2DM.
Results:
Most patients (182/206, 88.4%) demanded more immediate and comprehensive nutritional management and education. Both ChatGPT and GPT 4.0 passed the Chinese Registered Dietitian examination. ChatGPT's food recommendations were mainly in line with best practices, except for certain foods like root vegetables and dry beans. Professional dietitians' reviews of ChatGPT's responses to common questions were largely positive, with 162 out of 168 providing favorable reviews. The multilabel image recognition model evaluation showed that the Dino V2 model achieved an average F1 score of 0.825, indicating high accuracy in recognizing ingredients.
Conclusions:
The model evaluations were promising. The AI-based nutritionist program is now ready for a supervised pilot study.
Related Concept Videos
Diabetes Mellitus: Type 2 and Gestational
Diabetes: Management and Pharmacotherapy
Insulin remains the cornerstone of treatment for most patients with type 1 and many...
Carbohydrate Metabolism
Starch accounts for approximately 60% of the carbohydrates consumed by humans. Since amylase enzymes cannot function in the stomach's acidic environment, starch can only be digested in the mouth and small intestine. Simple sugars are found naturally in milk and fruits in...
Diabetes Mellitus: Overview and Type I Subtype
Type 1 diabetes is an autoimmune disease in which the immune system mistakenly attacks and destroys the insulin-producing beta cells in the pancreas. As a result, the body is unable to produce sufficient insulin, and individuals with...
Diabetes: Symptoms, Diagnosis, and Complications
Oral Hypoglycemic Agents: Biguanides and Glitazones

