Control of Magnetic Surgical Robots With Model-Based Simulators and Reinforcement Learning

Area of Science:

• Robotics
• Artificial Intelligence
• Medical Technology

Background:

• Magnetically manipulated medical robots offer miniaturization and tetherless actuation.
• Autonomous control is key for safe and accurate robotic surgery.
• Classical control methods struggle with complex, dynamic environments.

Purpose of the Study:

• To apply model-free reinforcement learning (RL) to magnetic needle manipulation.
• To overcome the impractical long runtimes of RL in real-world surgical robotics.
• To develop a faster, more accurate autonomous control method for medical robots.

Main Methods:

• Constructed a model-based simulation (MBS) using guided real-world exploration to learn environment dynamics.
• Applied RL within the MBS environment for intensive training.
• Transferred learned behaviors from MBS to the real-world system.

Main Results:

• The MBS approach achieved a 6 mm root-mean-square (RMS) error for a square trajectory.
• RL training in MBS was approximately 200 times faster than real-world training.
• Pure simulation-based methods resulted in a 31 mm RMS error, demonstrating poor transferability.

Conclusions:

• Model-based simulation (MBS) environments are effective for accelerating RL in domains where real-world training is impractical.
• This method significantly improves the speed and accuracy of autonomous control for magnetic medical robots.
• MBS provides a viable solution for developing robust control strategies when accurate simulations are initially unavailable.