A comparative study of vision-language models for food ingredient recognition and nutrient estimation

Area of Science:

• Computational nutrition and food science.
• Artificial intelligence applications in dietary assessment.

Background:

• Accurate food composition analysis is crucial for understanding nutritional and sensory properties.
• Traditional dietary assessment methods suffer from subjectivity and low reproducibility.
• Automated methods are needed to overcome limitations of manual food analysis.

Purpose of the Study:

• To explore the use of Vision-Language Models (VLMs) for automated food composition analysis.
• To evaluate VLM performance in food ingredient recognition and nutrient estimation.
• To introduce novel approaches for enhancing VLM accuracy in complex food scenarios.

Main Methods:

• Evaluation of state-of-the-art VLMs on the Nutrition5K dataset.
• Implementation of a progressive multi-view image recognition approach for ingredient recognition.
• Development of a prompting strategy using ingredient labels for nutrient estimation.

Main Results:

• VLMs demonstrated effectiveness in identifying primary food components from images.
• Challenges were observed in accurately quantifying nutrient content, especially for composite or ambiguous dishes.
• The progressive multi-view approach improved ingredient recognition sensitivity.

Conclusions:

• VLMs offer a promising avenue for AI-assisted food composition analysis.
• Current limitations in nutrient estimation for complex foods require further research.
• Future work should integrate chemical, visual, and computational perspectives for enhanced accuracy.