Co-localized augmented human and X-ray observers in collaborative surgical ecosystem

Area of Science:

• Medical Imaging
• Computer-Assisted Surgery
• Augmented Reality

Background:

• Image-guided percutaneous interventions offer safer alternatives to traditional surgery.
• Current image-guidance systems face challenges in associating 2D monitor images with 3D patient anatomy during complex procedures.

Purpose of the Study:

• To develop a collaborative augmented reality (AR) surgical ecosystem for precise co-localization of C-arm X-ray imaging and surgeon viewpoints.
• To enhance the spatial understanding and integration of imaging data within the surgical field.

Main Methods:

• Proposed a collaborative AR surgical ecosystem integrating C-arm X-ray and surgeon viewers.
• Implemented joint calibration of a C-arm scanner's visual tracker and X-ray source using a hand-eye calibration strategy.
• Utilized vision-based simultaneous localization and mapping (SLAM) for inside-out co-localization of observers in shared AR environments.

Main Results:

• Hand-eye calibration demonstrated convergence with 50 or more pose pairs, achieving mean translation and rotation errors of 5.7 mm and [Formula: see text], respectively.
• End-to-end target augmentation error studies showed a mean landmark distance of 10.8 mm between real and virtual environments.
• The system successfully provided a shared augmented experience for both human and X-ray viewers.

Conclusions:

• The developed AR system facilitates a shared augmented experience, improving spatial awareness for surgical teams.
• This collaborative AR solution has the potential to simplify surgeon hand-eye coordination and aid C-arm technologists in X-ray view-point planning.