A Study of Multi-Task and Region-Wise Deep Learning for Food Ingredient Recognition

Area of Science:

• Computer Vision
• Food Science
• Machine Learning

Background:

• Current food recognition systems primarily focus on dish categorization, neglecting ingredient composition, which impacts nutritional value.
• Dishes with identical names can vary significantly in ingredients and nutritional content.
• Limited availability of ingredient-labeled datasets hinders research in food ingredient recognition.

Purpose of the Study:

• To analyze key challenges in food ingredient recognition.
• To introduce a novel, large-scale dataset for food ingredient recognition research.
• To evaluate current approaches for ingredient recognition in food images.

Main Methods:

• Analysis of three core issues in ingredient recognition: image-level vs. region-level recognition, single vs. multiple image scales, and single vs. multi-task learning.
• Development and utilization of the Vireo Food-251 dataset, comprising 169,673 images of 251 Chinese food dishes and 406 ingredients.
• Evaluation of recognition performance on the new dataset to identify limitations of existing methods.

Main Results:

• The Vireo Food-251 dataset presents significant challenges in scale and complexity for current food ingredient recognition models.
• The analysis highlights the impact of different recognition strategies (image/region level, scale pooling, learning approaches) on performance.
• The study reveals the limitations of existing methods in accurately identifying food ingredients from images.

Conclusions:

• Ingredient recognition is crucial for accurate nutritional assessment and health applications.
• The proposed Vireo Food-251 dataset provides a valuable resource for advancing food ingredient recognition research.
• Further research is needed to develop more robust models capable of handling the complexities of ingredient identification in diverse food images.