Physically Informed 3D Food Reconstruction: Methods and Results

Area of Science:

• Computer Vision
• Nutrition Science
• 3D Reconstruction

Background:

• Accurate food portion size estimation is vital for nutrition analysis and dietary assessment.
• Current 3D reconstruction methods often prioritize surface geometry over volumetric accuracy.
• This limitation hinders precise portion size estimation essential for dietary studies.

Purpose of the Study:

• To introduce three novel methods (NS-DRS, HR-CMS, GS-GP) for real-scale 3D food reconstruction and volume estimation from limited 2D images.
• To address the challenge of volumetric inaccuracy in existing 3D reconstruction techniques for food analysis.
• To develop scalable tools for dietary monitoring and clinical nutrition.

Main Methods:

• A shared three-stage pipeline: 3D reconstruction, scale estimation using physical references (e.g., checkerboards), and mesh refinement.
• Three distinct methods differing in reconstruction backbones and scale-recovery mechanisms.
• Evaluation on diverse food items using single-view and multi-view images for volume and geometric accuracy.

Main Results:

• The methods demonstrate complementary strengths: NS-DRS excels in volume estimation, while GS-GP is superior in 3D reconstruction.
• All three methods achieved 18-23% lower volume estimation error compared to the state-of-the-art.
• Performance was assessed across various food textures, shapes, and camera poses.

Conclusions:

• Physically informed and explainable 3D reconstruction pipelines are effective for accurate food portion estimation.
• The developed methods offer improvements in both volumetric and geometric accuracy for 3D food models.
• These findings support the potential of these methods as scalable tools for dietary assessment and clinical nutrition.