Enhancing surgical navigation: a robust hand-eye calibration method for the Microsoft HoloLens 2

Area of Science:

• Medical Augmented Reality
• Surgical Navigation Systems
• Computer-Assisted Surgery

Background:

• Optical-see-through head-mounted displays (HMDs) offer potential for integrating virtual content into surgical settings.
• Clinical adoption of HMDs in surgery is limited by tracking inaccuracies and manual alignment requirements.

Purpose of the Study:

• To develop a simple, robust hand-eye calibration process for the Microsoft HoloLens 2 depth camera.
• To enable accurate integration of virtual surgical information with real-world anatomy.

Main Methods:

• A tracked surgical stylus with infrared reflective spheres was used as a calibration tool.
• Monte Carlo simulations and a paired-fiducial registration algorithm were employed.
• A user study validated the method in a clinical context.

Main Results:

• A calibration accuracy of 1.65 mm was achieved using as few as 6 fiducial points.
• User study participants achieved a mean accuracy of 1.67 mm in an average of 42 seconds.
• Heuristics for optimizing calibration accuracy were presented.

Conclusions:

• This work facilitates real-time hand-eye calibration for the Microsoft HoloLens 2, eliminating manual alignment.
• The developed framework allows seamless integration of existing surgical navigation systems into AR environments with high accuracy.