ImitateCholec: A Multimodal Dataset for Long-Horizon Imitation Learning in Robotic Cholecystectomy.

Area of Science:

• Robotics
• Surgical Technology
• Artificial Intelligence

Background:

• Global shortage of skilled surgeons necessitates advanced assistive technologies in operating rooms.
• Robotic surgery offers potential for enhanced precision and efficiency.
• Current autonomous systems require robust datasets for training complex surgical tasks.

Purpose of the Study:

• Introduce ImitateCholec, a novel dataset for advancing autonomous robotic surgery.
• Facilitate imitation learning for robotic systems in laparoscopic cholecystectomy.
• Enable development of robots capable of phase-level and procedural autonomy.

Main Methods:

• Collected over 18,000 demonstrations (approx. 20 hours) from 34 ex vivo porcine cholecystectomies.
• Segmented clipping and cutting phases into 17 distinct surgical tasks.
• Integrated multi-perspective endoscopic videos with da Vinci Research Kit kinematic data.

Main Results:

• Dataset includes optimal executions and recovery maneuvers to handle surgical variability.
• Enables training of imitation learning models for long-horizon surgical workflow execution.
• Supports applications in surgical workflow modeling, error recognition, and tool pose estimation.

Conclusions:

• ImitateCholec is a valuable resource for developing autonomous robotic surgical systems.
• The dataset advances the potential for robotic assistance in complex surgical procedures.
• Facilitates progress towards full procedural autonomy in robotic surgery.