Nail it! vision-based drift correction for accurate mixed reality surgical guidance

Area of Science:

• Medical Imaging
• Surgical Navigation
• Mixed Reality

Background:

• Head-mounted displays (HMDs) offer promising mixed reality (MR) guidance in surgery.
• Precise HMD tracking is critical, but current methods without fiducial markers suffer from significant drift.
• This drift leads to misaligned visualizations, hindering accurate surgical execution.

Purpose of the Study:

• To develop and validate a mixed reality surgical navigation workflow that continuously corrects for HMD drift using only image-based methods.
• To assess the feasibility and accuracy of this drift correction technique in glenoid pin placement for total shoulder arthroplasty.

Main Methods:

• A novel image-based drift correction workflow was implemented for HMDs.
• The system was tested using the Microsoft HoloLens for glenoid pin placement in total shoulder arthroplasty.
• Studies included phantom trials with multiple users and a cadaver trial performed by an attending surgeon.

Main Results:

• User satisfaction with registration overlay was high in both phantom and cadaver studies.
• Phantom study showed average errors of 1.5 mm (entry point) and 2.4° (orientation).
• Cadaver study demonstrated average errors of 2.5 mm (entry point) and 1.5° (orientation), outperforming native tracking.

Conclusions:

• Image-based drift correction enables precise alignment of mixed reality environments with patient anatomy.
• The developed technique facilitates highly accurate pin placement in shoulder arthroplasty.
• This represents a significant advancement toward markerless, purely image-based mixed reality surgical guidance.