Monocular suture needle pose detection using synthetic data augmented convolutional neural network.

Area of Science:

• Robotics and Automation
• Computer Vision
• Surgical Technology

Background:

• Robotic microsurgery demands high precision for delicate procedures.
• Accurate needle pose estimation is vital for robotic micro-suturing and autonomous control.
• Monocular vision presents challenges for precise needle pose estimation.

Purpose of the Study:

• To propose a convolutional neural network (CNN)-based method for suture needle pose estimation using monocular images.
• To enhance accuracy in robotic micro-suturing by optimizing insertion trajectories and enabling autonomous control.

Main Methods:

• A CNN was trained to detect keypoints (tip, middle, end) on the suture needle from 2D images.
• A hybrid dataset of real-world and synthetic images was utilized for model training.
• An algorithm was developed to estimate 3D keypoint positions and needle orientation.

Main Results:

• Experiments on synthetic data yielded low translation errors (0.098–0.118 mm) and orientation errors (12.75°–15.55°).
• Evaluation on real data showed average 2D translation errors of 0.047–0.052 mm.
• Over 93% of detected keypoints on real data had errors below 4 pixels.

Conclusions:

• The CNN-based method effectively estimates suture needle pose in monocular vision, utilizing synthetic data augmentation.
• The approach demonstrates promising performance on real-world data for robotic microsurgery applications.
• This technique holds potential for real-time pose estimation in automated surgical systems.