A Wearable Augmented Reality Navigation System for Surgical Telementoring Based on Microsoft HoloLens

Area of Science:

• Medical Technology
• Augmented Reality
• Surgical Innovation

Background:

• Effective surgical training is crucial for skill development.
• Telementoring can bridge geographical gaps in surgical expertise.
• Current AR systems offer potential but require refinement for clinical application.

Purpose of the Study:

• To develop and evaluate a novel augmented reality (AR) system for surgical training and telementoring.
• To integrate a Microsoft HoloLens device with a 3D point tracking module for real-time surgical guidance.
• To assess the system's performance and clinical applicability in simulated and in vivo settings.

Main Methods:

• Developed an AR system combining Microsoft HoloLens with a stereo camera for 3D scalpel tracking.
• Implemented virtual surgical scene superimposition for training and co-registration for telementoring.
• Validated system performance through benchtop experiments and in vivo testing in a rabbit model for skin grafting and fasciotomy.

Main Results:

• The AR system successfully tracked the scalpel's 3D position and provided virtual guidance.
• Benchtop experiments verified the system's performance characteristics.
• In vivo validation in a rabbit model demonstrated clinical applicability for telementored procedures.

Conclusions:

• The proposed AR system shows significant potential for enhancing surgical training and telementoring.
• It can improve surgical performance and reduce healthcare disparities, particularly in resource-limited remote areas.
• Further clinical integration is warranted to realize the full benefits of this technology.