Related Experiment Video
Updated: May 20, 2026
07:46
Automatic Surgery in Transcatheter Aortic Valve Replacement Using Augmented Reality
Published on: August 9, 2024
A Granular Interaction Framework for Surgical Robotic Digital Twins: Enabling Realistic Segment-Level Manipulation in
An-Nhien Vo1,2, Dhanakumaresh Subramani3,4, Joachim Schmidt3,4,5
1Department of Thoracic Surgery, University Hospital Bonn, Bonn, Nordrhein-Westphalia, Germany. an.vo@ukbonn.de.
Summary
This study introduces a new framework for integrating Robot-Assisted Surgery (RAS) into Extended Reality (XR) training using scan-derived digital twins (DTs) and a force-based interaction system for enhanced realism.
Area of Science:
- Robotics
- Surgical Simulation
- Extended Reality
Background:
- Robot-Assisted Surgery (RAS) adoption is limited by steep learning curves.
- Extended Reality (XR) and Digital Twins (DTs) offer potential solutions for surgical training.
- Current integration methods need high-fidelity simulations for complex kinematic constraints.
Purpose of the Study:
- To propose a modernized framework for integrating RAS into XR.
- To transition from CAD models to scan-derived DTs for improved fidelity.
- To implement a force-based interaction paradigm for realistic surgical simulation.
Main Methods:
- Performance profiling of Apple Vision Pro (AVP) for hardware benchmarks.
- Developing a Dexter Surgical Robotic System surrogate using 3D scanning and retopology.
- Evaluating Rigidbody vs. ArticulationBody physics and developing a force-based interaction system within Unity.
Main Results:
- Scan-derived model achieved 1.6mm spatial accuracy.
- ArticulationBody physics offered superior numerical stability over Rigidbody.
- Geometric constraint controller provided high-speed stability without calibration, unlike the linear feedback controller.
Conclusions:
- The framework modernizes RAS integration with a non-CAD workflow.
- A prescriptive testing protocol for mesh decimation was established.
- A force-based interaction system enhances kinematic realism for clinical training.
