Fruit-stem structural visual perception for picking point localization in strawberry harvesting robots

Area of Science:

• Robotics and Automation
• Computer Vision
• Agricultural Engineering

Background:

• Strawberry harvesting faces challenges in picking point localization due to fruit-stem scale variation, occlusion, and detection difficulties.
• Accurate localization is crucial for automated harvesting systems to improve efficiency and reduce fruit damage.

Purpose of the Study:

• To develop an improved fruit-stem oriented detection and picking point localization method for ridge-cultivated strawberries.
• To enhance the accuracy and robustness of visual perception systems for strawberry harvesting robots.

Main Methods:

• An enhanced YOLOv11n-OBB model (FSGE-OBB) incorporating a Feature Fusion Lite Module (FFLM) and Stem Direction Enhancement Module (SDEM).
• Integration of Grouped Spatial Excitation Convolution (GSEConv) and Efficient Upsampling Convolution Block (EUCB) to optimize feature representation and detail recovery.
• A geometry-constrained fruit-stem association method based on oriented bounding boxes (OBBs) for picking point localization.

Main Results:

• The FSGE-OBB model achieved mAP@0.5 of 78.85% and mAP@0.5:0.95 of 68.27%, outperforming the baseline.
• The fruit-stem association method demonstrated a success rate of 97.99% with low mean absolute errors in 3D localization (2.2-2.6 mm).
• Simulated harvesting achieved a 93.3% success rate with an average time of 9.2 s per fruit.

Conclusions:

• The proposed FSGE-OBB framework effectively unifies fruit-stem detection, association, and picking point localization for strawberries.
• This method offers a feasible and effective visual perception solution for automated strawberry harvesting robots.
• The advancements address key challenges in robotic harvesting, paving the way for practical implementation.