Polygon-Aware Deep Learning Framework for Meal-Level Nutrition Estimation From Food Images

Area of Science:

• Computer Vision
• Nutritional Science
• Machine Learning

Background:

• Automated dietary assessment relies on accurate nutritional content estimation from food images.
• Existing computer vision methods often use coarse bounding boxes, limiting quantitative nutrition estimation.
• Precise food segmentation is crucial for detailed analysis.

Purpose of the Study:

• To develop a polygon-aware, instance-level framework for precise meal nutrition estimation.
• To integrate fine-grained food segmentation with region-specific feature extraction.
• To enhance the accuracy of automated dietary assessment systems.

Main Methods:

• Employed a YOLOv8n-based instance segmentation model for food item localization.
• Utilized polygon-aware extraction of shape, color, and texture features from segmented food regions.
• Aggregated instance-level features using area-weighted pooling for meal-level representation.
• Applied multiple regression models (Random Forest, XGBoost, LightGBM, Ridge, CNN) for nutrition estimation.

Main Results:

• Achieved mask mAP@0.5 of 0.4232 and mAP@0.5:0.95 of 0.3400.
• Demonstrated high qualitative overlap with a mean IoU of 0.9434 and F1-score of 0.87 for instance-level classification.
• Polygon-aware features consistently improved nutrition estimation by an average of 11.63%, with XGBoost performing best.
• Statistical significance testing confirmed robust improvements (p < 0.05 for most nutrients).

Conclusions:

• Fine-grained, polygon-level representations are effective for reliable and explainable image-based nutrition estimation.
• The proposed framework offers a scalable foundation for real-world dietary monitoring applications.
• This method can assist consumers and professionals in assessing meal composition via mobile devices.