A Marker-Less Registration Approach for Mixed Reality-Aided Maxillofacial Surgery: a Pilot Evaluation

Area of Science:

• Medical Imaging and Augmented Reality
• Surgical Navigation Systems
• Computer-Assisted Surgery

Background:

• Current tumor resection protocols rely on intraoperative tissue examination, potentially extending surgery duration and operating room occupancy.
• Augmented reality (AR) shows promise in surgery but often requires manual or marker-based registration.
• Existing methods for integrating medical imaging into surgical workflows can be time-consuming and resource-intensive.

Purpose of the Study:

• To develop and evaluate a marker-less augmented reality application for registering Positron Emission Tomography-Computed Tomography (PET-CT) data in surgical tumor resections.
• To assess the accuracy of the marker-less system against a marker-based approach.
• To gather feedback from surgical specialists on the application's usability and potential.

Main Methods:

• An application was designed to combine facial landmarks from an RGB video stream with the Spatial-Mapping API of the Microsoft HoloLens headset for marker-less registration.
• PET-CT information was registered to the patient's anatomy without the use of physical markers.
• System accuracy was quantified by comparing it to a marker-based registration method, and specialist opinions were collected via an ISO-9241/110-based survey.

Main Results:

• The marker-less system demonstrated an average positioning error of (x, y, z) = (3.3 ± 2.3, -4.5 ± 2.9, -9.3 ± 6.1) mm.
• This represents an approximate 3 mm increase in positioning error along the x and y axes compared to the marker-based approach, potentially due to the lack of explicit markers.
• Surgical specialists positively evaluated the application, expressing interest in further development and providing constructive criticism.

Conclusions:

• The developed marker-less AR application offers a viable alternative for integrating PET-CT data into surgical tumor resections.
• While accuracy is slightly reduced compared to marker-based systems, the system shows potential for improving surgical workflows.
• Further research and development are warranted based on positive specialist feedback and identified areas for improvement.