Integration of Reinforcement Learning in a Virtual Robotic Surgical Simulation

Area of Science:

• Artificial Intelligence
• Surgical Simulation
• Robotics

Background:

• AI advancements offer potential to enhance surgical capabilities.
• Integrating deep learning with high-fidelity surgical simulation presents challenges.
• This research explores reinforcement learning for automated surgical maneuvers.

Purpose of the Study:

• To develop and evaluate a reinforcement learning (RL) system for automating surgical maneuvers in a graphical simulation environment.
• To investigate the effectiveness of different reward functions in training autonomous surgical agents.
• To assess the feasibility of using game engines for surgical simulation and AI integration.

Main Methods:

• Utilized the Unity3D game engine with the Machine Learning-Agents package and NVIDIA FleX particle simulator.
• Developed RL-trained virtual scissors using Proximal Policy Optimization (PPO).
• Tested constant and proportional reward functions, employing TensorFlow for analysis and tuning.

Main Results:

• RL-trained scissors demonstrated reliable manipulation of simulated soft tissue.
• Achieved a desirable cutting trajectory along one axis.
• Proportional rewards outperformed constant rewards; performance on two-axis movement was inconsistent.

Conclusions:

• Game engines are a promising platform for developing RL-based solutions for simulated surgical tasks.
• One-dimensional movement tasks were successfully completed in simulations.
• Further optimization of network architecture and parameter tuning is required for complex tasks.