De-occlusion models and diffusion-based data augmentation for size estimation of on-plant oriental melons

Area of Science:

• Agricultural Science
• Computer Vision
• Plant Phenotyping

Background:

• Accurate fruit size estimation is vital for precision agriculture and crop management.
• Leaf occlusion presents a significant challenge in estimating fruit size in vertical cultivation systems.

Purpose of the Study:

• To estimate the size of on-plant oriental melons, addressing leaf occlusion challenges.
• To evaluate the performance of instance segmentation and de-occlusion models for fruit size estimation.

Main Methods:

• Utilized a diffusion model for data augmentation by generating synthetic occluding leaves.
• Implemented and compared three instance segmentation models (Mask R-CNN, Mask2Former, DETR) and six derived de-occlusion models.
• Assessed model performance using average precision, mean absolute error, and mean absolute percentage error.

Main Results:

• Amodal Mask2Former and occlusion-aware RCNN (ORCNN) achieved high average precision scores (85.92% and 85.35%).
• Amodal Mask2Former demonstrated superior performance in size estimation with low mean absolute errors (5.46 mm height, 4.20 mm diameter).
• Transformer-based models outperformed CNN architectures in de-occlusion and size estimation tasks across various occlusion ratios (0-70%).

Conclusions:

• Transformer-based Amodal Mask2Former shows enhanced capabilities for fruit de-occlusion and size estimation compared to CNNs.
• The developed de-occlusion models significantly improve fruit size estimation accuracy in the presence of occlusion.
• Generating synthetic datasets with over 70% occlusion remains a limitation for current de-occlusion models.