IngredSAM: Open-World Food Ingredient Segmentation via a Single Image Prompt

Area of Science:

• Computer Vision
• Artificial Intelligence
• Food Image Analysis

Background:

• Food semantic segmentation is crucial for AI applications like food image analysis.
• Traditional supervised methods struggle with the complexity and variety of food ingredients.
• Existing models face challenges with diverse appearances and class imbalances.

Purpose of the Study:

• To introduce IngredSAM, a novel approach for open-world food ingredient semantic segmentation.
• To extend the capabilities of the Segment Anything Model (SAM) for food analysis.
• To address limitations of supervised methods in food ingredient segmentation.

Main Methods:

• Utilized visual foundation models (VFMs) and prompt engineering.
• Leveraged discriminative and matchable semantic features from clean image prompts.
• Applied one-shot, open-world segmentation without a training process.

Main Results:

• Achieved significant advancements in food ingredient segmentation.
• Demonstrated superior performance on FoodSeg103 and UECFoodPix datasets, outperforming state-of-the-art methods by 2.85% and 6.01% respectively.
• Successfully generated accurate segmentation masks in real-world scenarios.

Conclusions:

• IngredSAM represents a successful application of one-shot, open-world segmentation for food ingredients.
• The framework offers accurate segmentation without requiring a training phase.
• Paves the way for enhanced nutritional analysis and consumer dietary trend monitoring.