Nail It! A learning framework for autonomous surgical suturing and teleoperation on the dVRK

Area of Science:

• Robotics
• Surgical Simulation
• Machine Learning

Background:

• Automating surgical suturing requires advanced simulation tools for skill acquisition.
• High-fidelity, readily available learning frameworks for robotic surgery are scarce, particularly for the da Vinci Research Kit (dVRK).

Purpose of the Study:

• To introduce Nail It!, a comprehensive Unity-ROS-dVRK framework for robotic surgery.
• To support both teleoperation data collection and Reinforcement Learning (RL) training of surgical skills like needle grasping and placement.

Main Methods:

• Developed a physics-based Unity environment with accurate dVRK kinematic modeling.
• Integrated real-time ROS communication for surgeon control and an intuitive GUI for tuning and algorithm development.
• Implemented RL training using Proximal Policy Optimization (PPO) with curriculum learning and domain randomization for robust policy training.

Main Results:

• Nail It! successfully trained autonomous suturing policies using PPO with curriculum learning and domain randomization.
• Policies achieved 95% accuracy in simulation for multi-step suturing procedures.
• Demonstrated robustness to visual and positional variations up to 15 mm.

Conclusions:

• Nail It! offers a modular, high-fidelity platform for developing and benchmarking autonomous surgical skills.
• The framework integrates accurate kinematics, RL, and teleoperation for autonomous training and human-in-the-loop control.
• Direct ROS connectivity and GUI facilitate rapid development and sim-to-real experimentation.