3D Camera and Single-Point Laser Sensor Integration for Apple Localization in Spindle-Type Orchard Systems

Area of Science:

• Agricultural Robotics
• Computer Vision
• Sensor Fusion

Background:

• Automated apple harvesting requires precise apple localization, which is challenging outdoors due to variable lighting affecting 3D cameras.
• Existing methods struggle with depth sensing accuracy in dynamic outdoor environments, hindering robotic harvesting efficiency.

Purpose of the Study:

• To develop and evaluate an integrated sensor system that overcomes light variations for accurate apple localization in outdoor robotic harvesting.
• To enhance the positional accuracy of robotic harvesting systems under diverse lighting conditions.

Main Methods:

• Integrated a RealSense D455f RGB-D camera with a single-point laser ranging sensor for precise apple localization.
• Utilized the EfficientDet object detection algorithm (mAP@0.5 = 0.775) and DeepSORT tracking for real-time apple identification and positioning.
• Mounted the combined sensor system on a tractor for experiments in indoor and outdoor artificial apple orchard environments.

Main Results:

• The integrated sensor system demonstrated significantly lower root-mean-square error (RMSE) values (1.62–2.13 cm) compared to the RGB-D camera alone (3.91–8.36 cm) across various light conditions.
• Achieved a positional accuracy of ±2 cm for the apple harvesting robotic manipulator, outperforming traditional depth sensing methods.
• Identified limitations with occluded apples due to leaves and branches, indicating areas for future improvement.

Conclusions:

• The integrated single-point laser and RGB-D camera system effectively mitigates outdoor light variations, enabling accurate apple localization for robotic harvesting.
• This approach offers a robust solution for enhancing the precision and reliability of automated agricultural operations.
• Further research will focus on addressing occlusion issues for comprehensive harvesting automation.