Autonomic Robotic Ultrasound Imaging System Based on Reinforcement Learning

Area of Science:

• Robotics
• Medical Imaging
• Artificial Intelligence

Background:

• Traditional ultrasound (US) imaging requires skilled operators and can be challenging for soft, moving targets.
• Marker-less tracking and real-time control are critical for advanced robotic surgery applications.

Purpose of the Study:

• To develop an autonomous robotic ultrasound (US) imaging system using reinforcement learning (RL).
• To enable fully autonomous imaging of soft, moving, and marker-less targets using only RGB images.
• To achieve real-time US probe control, constant force tracking, and automatic imaging.

Main Methods:

• A state representation model was developed to encode force and US information into image space.
• A reinforcement learning (RL) agent was trained using a policy gradient theorem with single RGB images as input.
• A force-to-displacement control method using an admittance controller was proposed for adaptable constant force tracking.

Main Results:

• Simulation experiments verified the feasibility of the integrated autonomous imaging method.
• The force-to-displacement control method demonstrated safe and effective adaptable constant force tracking.
• Phantom and volunteer experiments confirmed the system's feasibility on real-world applications.

Conclusions:

• The developed approaches are stable and feasible for autonomic and accurate control of the US probe.
• The autonomous robotic US imaging system shows significant potential for image-guided and robotic surgery.