Related Experiment Video
Updated: Aug 5, 2026
07:46
Automatic Surgery in Transcatheter Aortic Valve Replacement Using Augmented Reality
Published on: August 9, 2024
Digital Twin-Based Virtual Reality Framework for Interaction and AI-Assisted Control of a Parallel Surgical Robot
Florin Covaciu1,2, Nadim Al Hajjar3, Anca-Elena Iordan2,4
1Research Center for Industrial Robots Simulation and Testing-CESTER, Technical University of Cluj-Napoca, 400114 Cluj-Napoca, Romania.
Sensors (Basel, Switzerland)
|July 28, 2026
Summary
This study introduces a Digital Twin-based virtual reality system for robot-assisted surgery training and control. The integrated AI enhances surgical robot performance and user interaction for improved outcomes.
Area of Science:
- Robotics
- Medical Technology
- Virtual Reality
Background:
- Robot-assisted minimally invasive surgery (RAMIS) requires integrated training and operational frameworks.
- Existing systems often lack seamless integration between virtual simulation and physical robotic control.
Purpose of the Study:
- To develop a Digital Twin-based virtual reality (VR) system for an innovative parallel surgical robot.
- To support both surgical training and real-time robot interaction within a unified platform.
Main Methods:
- A bidirectional Digital Twin architecture synchronizing a VR environment with the ATHENA parallel robot (4-DOF, RCM constraint).
- Integration of VR controllers for intuitive manipulation and an AI module (neural networks) for trajectory refinement and motion guidance.
- Deployment of the AI model using an ONNX-compatible runtime for efficient inference.
Main Results:
- Experimental validation demonstrated improvements in motion stability, task execution efficiency, and user interaction quality.
- The system maintained a high level of control intuitiveness during surgical tasks.
- External motion assessment confirmed the system's performance.
Conclusions:
- Digital Twin-based VR systems offer a unifying platform for surgical training, interaction, and intelligent assistance.
- The developed system enhances next-generation medical robotic capabilities.
- The integration of AI and VR with robotic systems shows significant potential in surgery.
