Learning intraoperative organ manipulation with context-based reinforcement learning.

Area of Science:

• Robotics
• Artificial Intelligence
• Surgical Technology

Background:

• Robotic surgery automation faces challenges due to high variability in surgical scenes.
• Generalizing automation solutions across different surgical contexts and patient variations is difficult.
• The Pneumatic Attachable Flexible (PAF) rail is a novel tool for robotic-assisted organ manipulation.

Purpose of the Study:

• To develop a novel reinforcement learning (RL) framework, rlman, for robotic surgery manipulation skills.
• To train contextual RL agents to address various aspects of the pick-and-place task using the PAF rail.
• To evaluate the framework's ability to support diverse RL algorithms and assess generalization capabilities.

Main Methods:

• Built upon an open-source surgical RL training environment to create the rlman framework.
• Implemented rlman to support both low- and high-dimensional state information for surgical sub-tasks in simulation.
• Trained state-of-the-art RL agents using rlman on four distinct surgical sub-tasks involving manipulation skills with the PAF rail.

Main Results:

• Achieved successful training of RL agents for four surgical sub-tasks using the rlman framework and PAF rail.
• Compared performance against state-of-the-art benchmarks, demonstrating competitive results.
• Evaluated and confirmed the agent's ability to generalize across different aspects of the surgical environment.

Conclusions:

• The rlman framework effectively supports training various RL algorithms for surgical sub-tasks.
• Contextual information is crucial for enhancing the generalization capabilities of surgical automation.
• Future work includes deploying trained policies on the da Vinci surgical system via dVRK for real-world validation.