GANana: Unsupervised Domain Adaptation for Volumetric Regression of Fruit

Area of Science:

• Computer Vision
• 3D Reconstruction
• Machine Learning

Background:

• 3D reconstruction is crucial for fruit grading and size estimation.
• A major challenge is the scarcity of labeled training data, especially for real-world applications.
• Existing methods often rely on extensive datasets of high-quality image-model pairs.

Purpose of the Study:

• To develop an unsupervised domain adaptation approach for 3D fruit reconstruction.
• To overcome the limitation of labeled data by utilizing synthetic and unlabeled real-world datasets.
• To improve the performance of 3D reconstruction on real fruit images.

Main Methods:

• Utilized volumetric regression for 3D reconstruction.
• Generated a synthetic dataset of 25,000 image-volume pairs of bananas using Blender.
• Employed a Generative Adversarial Network (GAN) with a volumetric consistency loss to adapt synthetic images to the real-world domain.
• Supplemented training with unlabeled real-world image datasets.

Main Results:

• Achieved improved 3D reconstruction performance on real-world images.
• Demonstrated the effectiveness of unsupervised domain adaptation in bridging the synthetic-to-real data gap.
• Successfully generated 3D banana reconstructions from single images.

Conclusions:

• The proposed method effectively leverages synthetic data while maintaining good performance on real-world images.
• This approach offers cost-effective 3D reconstruction solutions by reducing reliance on extensive real-world labeled data.
• The method is generalizable to various 3D reconstruction tasks in plant phenotyping and beyond.