Barrier-free tomato fruit selection and location based on optimized semantic segmentation and obstacle perception...

Area of Science:

• Robotics
• Computer Vision
• Agricultural Technology

Background:

• Vision-based fruit detection is crucial for harvesting robots.
• Obstacles like stems and wires often obscure fruits, hindering accurate identification.
• Existing methods lack robust obstacle perception for precise fruit selection.

Purpose of the Study:

• To develop an advanced fruit target selection and location approach for harvesting robots that incorporates obstacle perception.
• To enhance the accuracy of fruit detection in complex greenhouse environments by addressing occlusion issues.

Main Methods:

• Generated synthetic data for tomato harvesting using 3D simulation and semantic segmentation.
• Designed an attention-based spatial-relationship feature extraction module (SFM) to improve DeepLab v3+ segmentation of linear obstacles.
• Developed an adaptive K-means clustering for fruit instance segmentation and a barrier-free fruit selection algorithm.

Main Results:

• Improved semantic segmentation achieved 96.75% accuracy, with IoU for wires and stems increasing by 5.0% and 2.3%.
• Obstacle type identification reached 96.15%, and occluded fruits were excluded with 86.67% effectiveness.
• The proposed method showed an 18.9% increase in selection precision and a 1.3% reduction in location error compared to Yolo v5.

Conclusions:

• The enhanced semantic segmentation and obstacle perception algorithms effectively address fruit occlusion by stems and wires.
• The barrier-free fruit selection algorithm reliably identifies optimal picking targets in complex environments.
• This approach offers a more practical and accurate solution for fruit harvesting robots, applicable to various crops.