A two-dimensional axis estimation method for pickable canopy apples based on YOLO cascade network

In a complex orchard environment, canopy apples are obscured by various factors, making it hard for apple harvesting robots to accurately determine which apples can be directly harvested. Furthermore, the complex obstruction leads to difficulties in identifying keypoints on the apples and caculating the axis direction, directly affecting the robot's determination of grasping positions.

To solve these issues, a two-dimensional (2D) axis estimation method for pickable canopy apples based on a YOLO cascade network was proposed. Firstly, the study introduced SPDConv for lossless downsampling and adopts dynamic upsampling to improve segmentation boundary accuracy, constructing an instance segmentation network named the YOLO-SD model to select pickable apples according to different occlusion conditions and growth states. Secondly, the geometric center of the mask image was located using its minimum enclosing circle, and the region of interest was extracted through morphological dilation. Then, by integrating the RFAConv, SCSA attention, and MBConv modules, a keypoint detection network YOLO-RSM was constructed to extract keypoints of pickable apples. Finally, a 2D axis construction strategy was proposed, which adaptively constructs the growth axis based on the visibility of keypoints.

Experimental results show that the overall average accuracy mAP50 of the YOLO-SD model for apple segmentation reached 95.2%, and the parameter quantity was reduced to 2.47 M. The average accuracy of the YOLO-RSM model for keypoint detection has reached 90.3%, which is 2.4%, 2.6%, and 4.7% higher than that of the YOLOv8n, YOLO11n, and YOLO12n models respectively. The 2D axis estimation algorithm has an average axis angular error of 7.23° ± 16.73°, and an axis estimation accuracy of 92.68%.

The proposed method can achieve high-precision canopy apple segmentation, keypoint detection, and 2D axis estimation, thus offering technical support for the picking operations of apple harvesting robots.