The IoRT-in-Hand: Tele-Robotic Echography and Digital Twins on Mobile Devices

Area of Science:

• Robotics
• Telemedicine
• Internet of Robotic Things (IoRT)
• 5G/6G Networks

Background:

• Telemedicine faces challenges in remote areas, conflicts, and disasters due to specialist scarcity and safety risks.
• Safe physical interaction with robotic systems in non-clinical environments is a significant hurdle for remote healthcare.

Purpose of the Study:

• To present a novel IoRT approach for remote healthcare, enabling specialist-to-robotic hand interaction.
• To introduce the IoRT-in-hand end-effector and an associated open-source Android application for real-time remote manipulation.

Main Methods:

• Developed the IoRT-in-hand: a smart end-effector with integrated sensors, camera, and connectivity.
• Created an open-source Android app using MQTT and ROS for remote control.
• Implemented an Edge-Cloud architecture with a Digital Twin for precise control and 3D feedback.
• Conducted a proof-of-concept tele-robotic ultrasound scan over 2300 km via a 5G network.

Main Results:

• Demonstrated successful teleoperation of a 6-DOF manipulator using the IoRT-in-hand system.
• Achieved precise control and 3D visual feedback in a remote ultrasound procedure.
• Performance validated through human feedback, sensory data, and latency measurements.

Conclusions:

• The IoRT-in-hand system shows significant potential for remote healthcare and emergency applications.
• Open-source availability of code and models encourages reproducibility and further development in robotic telemedicine.