Boosting RGB-D Pear Detection via Depth-Constraint Enhanced Gaussian Prior.

Area of Science:

• Computer Vision
• Agricultural Robotics
• Machine Learning

Background:

• Accurate pear detection is crucial for automated harvesting but is hindered by occlusion, overlapping fruits, cluttered backgrounds, and variable lighting.
• Existing RGB-D methods often underutilize depth information and do not account for the elliptical shape of pears.

Purpose of the Study:

• To develop a multimodal pear detection framework that effectively integrates RGB and depth data.
• To improve the accuracy and robustness of pear detection in complex orchard environments.
• To enhance the potential for practical deployment in automated pear harvesting systems.

Main Methods:

• A Siamese convolutional backbone and Transformer-based fusion architecture are employed to jointly model RGB and depth information.
• Gaussian Prior Boxes are introduced for more precise localization, aligning better with pear contours.
• A Depth-Aware Constraint enforces depth consistency, improving robustness in cluttered scenes.
• A Robust Cross-Modal Token Exchange strategy enhances feature interaction between modalities during training.

Main Results:

• The proposed framework achieved an AP50 of 0.961, precision of 0.941, recall of 0.951, and F1-score of 0.942 on pear detection.
• Demonstrated significant improvements over YOLOv8-l RGB (+4.3 AP50) and YOLOv8-l RGB-D (+2.9 AP50) baselines.
• Achieved AP50 of 0.927 on the KFuji apple dataset, surpassing the state-of-the-art.
• Operates at 41.2 FPS, indicating a strong balance between accuracy and real-time performance.

Conclusions:

• The multimodal framework effectively leverages RGB and depth data for superior pear detection.
• The proposed components (Gaussian Prior Boxes, Depth-Aware Constraint, Cross-Modal Token Exchange) significantly enhance detection performance and robustness.
• The method shows strong potential for practical application in automated harvesting systems due to its accuracy and efficiency.