Related Experiment Video
Updated: May 19, 2026
Pedicle Screw Placement Using an Augmented Reality Head-Mounted Display in a Porcine Model
Published on: May 24, 2024
Markerless 3D ultrasound reconstruction for augmented reality assisted minimally invasive spine surgery
Ruixuan Li1, Yuyu Cai2, Ayoob Davoodi2
1Department of Mechanical Engineering, Robot-Assisted Surgery Group, KU Leuven, Leuven, Belgium. ruixuan.li@kuleuven.be.
Purpose:
Minimally invasive spine surgery (MISS) requires 3D navigation of bony anatomy, yet conventional navigation is limited by radiation or line-of-sight-dependent external trackers. This study developed a markerless augmented reality ultrasound (AR-US) system to provide radiation-free 3D visualization while eliminating tracker occlusion.
Methods:
An integrated pipeline was developed to track a handheld US probe using a head-mounted display (HMD). The probe pose was estimated using FoundationPose during scanning. To mitigate inherent noise in the raw estimates, an Extended Kalman Filter was implemented for refinement. Spatially tracked 2D US images were subsequently used to reconstruct a 3D volume, which was rendered on the HMD.
Results:
The proposed system reconstructed 3D volumes with an average representation error of mm on different experimental phantoms. The 3D visualization was also successfully rendered on the HMD, qualitatively demonstrating a spatial alignment with key anatomic features on a spine phantom.
Conclusion:
This work demonstrates the feasibility of a markerless AR-US system for coarse navigation in MISS. The presented approach offers a streamlined and intuitive solution, providing clinicians with 3D anatomy for navigating instrumentation..
