Robotic ultrasound scanning platform with autonomous control, multimodal human-machine interface and real-time image...

Autonomous robots can streamline repetitive and time-consuming surgical tasks like ultrasound scanning. AI can provide the necessary intelligence, but for clinical acceptance, systems must move predictably, offer intuitive interaction, and maintain low latency on medical-grade hardware.

We present an integral robotic platform for autonomous ultrasound scanning. The platform allows to perform scanning on either manual or autonomous mode. The autonomous control combines AI-driven ultrasound target segmentation with geometric motion algorithms designed to focus the probe on the segmented target. We optimized the image processing models for specialized hardware to ensure real-time performance with constrained resources. Users can control the platform via natural language voice commands processed by a Large Language Model (LLM) and visualize the procedure through a synchronized 3D Digital Twin and an augmented reality (AR) environment.

We tested the platform in hepatic tumor localization in a synthetic phantom. Autonomous tumor localization showed a success rate of 84% with an average execution time of 5.75 s.

Our system successfully automates ultrasound tasks, allowing the user to control the process through an intuitive, multimodal interface. By pairing optimized AI with specialized hardware, we achieved low-latency, real-time adaptability with minimal hardware resources. In the future we plan to adapt this platform for more complex neurosurgical and urological procedures.