Visual nutrition analysis: leveraging segmentation and regression for food nutrient estimation.

Area of Science:

• Computer Vision
• Nutritional Science
• Machine Learning

Background:

• Accurate nutritional assessment is vital for health and disease prevention.
• Existing image-based food analysis methods struggle with precise nutritional content prediction due to reliance on volume estimation and lack of detailed labels.
• Consumers often lack the systematic knowledge to accurately determine food's nutritional value from images.

Purpose of the Study:

• To develop an accurate image-based model for direct prediction of nutritional content in dishes.
• To overcome limitations of current methods by integrating segmentation and regression tasks.
• To provide a tool for easier and more precise dietary assessment using food images.

Main Methods:

• Manual segmentation annotation was performed on the Nutrition5k dataset.
• A UNet model was employed for initial food segmentation.
• A backbone network with Squeeze-and-Excitation structure was used for feature extraction.
• Fully connected layers processed features for predicting weight, calories, fat, carbohydrates, and protein content.

Main Results:

• The model achieved an average Percentage Mean Absolute Error (PMAE) of 17.06% for predicted nutritional components.
• Successfully combined segmentation and regression for improved nutritional prediction accuracy.
• Provided manually annotated segmentation labels for public use.

Conclusions:

• The proposed segmentation-first, regression-after approach effectively predicts nutritional content from food images.
• This method offers a significant advancement in automated dietary assessment technology.
• The availability of annotated data facilitates further research in image-based nutritional analysis.