A reproducible framework for synthetic data generation and instance segmentation in robotic suturing.

Area of Science:

• Robotics and Computer Vision
• Surgical Automation
• Medical Imaging

Background:

• Automating suturing in robotic surgery enhances precision and efficiency but is limited by scarce, complex real-world surgical data.
• Developing robust computer vision (CV) models for this task is challenging due to data acquisition and annotation difficulties.

Purpose of the Study:

• To address the scarcity of surgical datasets for robotic suturing automation using a sim-to-real approach.
• To develop and evaluate data-driven methodologies for training and validating CV models on synthetic and real surgical data.

Main Methods:

• Generated three synthetic datasets with increasing realism using 3D models in Unity for training object detection models.
• Evaluated the generalizability of synthetic-trained models and the impact of dataset realism on performance.
• Developed a hybrid training strategy combining synthetic and minimal real data for a real-time instance segmentation model.

Main Results:

• Increasing synthetic dataset realism improved model performance on real data, but full generalization was not achieved.
• The hybrid approach significantly enhanced performance in real-world scenarios.
• The hybrid instance segmentation model achieved real-time performance with a Dice coefficient of 0.92 using only 30-50 real images.

Conclusions:

• Sim-to-real synthetic datasets offer a viable framework for advancing robotic suturing automation.
• Shared resources (3D models, environments, datasets) facilitate further research, dataset expansion, and model fine-tuning.
• This work provides a foundation for addressing dataset scarcity and improving suturing automation in robotic surgery.