Building a hybrid patient's model for augmented reality in surgery: a registration problem

Area of Science:

• Medical Imaging
• Computer-Assisted Surgery
• Augmented Reality

Background:

• Merging diverse data into a hybrid patient model is crucial for augmented reality (AR) in surgery.
• This integration presents significant technical challenges, particularly in data registration.

Purpose of the Study:

• To develop a concept for merging patient data with the surgical environment.
• To address the technical problem of registration between various sensor data streams.
• To present a general framework for data registration in surgical AR.

Main Methods:

• Literature review of the state-of-the-art in surgical AR registration.
• Development of a general framework for registration of sensor data.
• Utilizing anatomical reference surfaces for registration.
• Employing internal patient structures when external references are insufficient.

Main Results:

• Demonstrated a framework for merging patient data and surgical reality.
• Showcased registration methods using anatomical reference surfaces.
• Validated the necessity of internal patient structures for registration in many clinical scenarios.

Conclusions:

• Successful implementation of hybrid patient models in AR surgery relies on robust data registration.
• Internal anatomical structures are often essential for accurate registration, especially in complex clinical cases.
• The presented framework offers a pathway for advancing AR applications in surgical settings.