Food Detection and Segmentation from Egocentric Camera Images

Area of Science:

• Computer Vision
• Artificial Intelligence
• Wearable Technology

Background:

• Accurate food intake monitoring is crucial for understanding eating habits and managing health conditions.
• Wearable sensors, particularly cameras, offer a feasible method for automatic food image capture.
• Automated food detection is essential for analyzing unstaged, real-world images from egocentric cameras.

Purpose of the Study:

• To develop and evaluate a novel pipeline for food detection and segmentation from egocentric wearable camera images.
• To accurately identify and localize food items within complex, real-world visual scenes.
• To segment food items for detailed nutritional analysis and intake pattern recognition.

Main Methods:

• An ensemble of YOLOv5 detection networks was trained to detect and localize food, beverages, screens, and persons.
• The model achieved a mean average precision of 80.6% across the four object categories.
• The Normalized-Graph-Cut algorithm was employed for segmenting detected food objects, achieving an average IoU of 82% for sharp images.

Main Results:

• The YOLOv5 ensemble effectively detected and localized food items with high accuracy.
• Food segmentation using Normalized-Graph-Cut demonstrated a significant average IoU of 82%.
• The system successfully processed images from wearable cameras for food intake analysis.

Conclusions:

• Automated food detection and segmentation from wearable cameras is a viable approach for efficient food intake monitoring.
• This technology has significant clinical relevance for preventing and treating eating disorders, obesity, and malnutrition.
• Personalized nutritional adjustments and healthy lifestyle guidance can be facilitated by understanding dietary patterns through this system.