Pose Estimation and Non-Rigid Registration for Augmented Reality During Neurosurgery

Area of Science:

• Neurosurgery
• Medical Imaging
• Computer-Assisted Surgery

Background:

• Craniotomy for brain tumor treatment can be complicated by tissue deformation.
• Accurate pre-operative imaging registration is crucial for successful neurosurgical interventions.

Purpose of the Study:

• To develop a novel Augmented Reality (AR) neurosurgical system.
• To accurately superimpose pre-operative 3D MRI-derived models onto the intra-operative surgical view.
• To address challenges posed by brain tissue deformation during surgery.

Main Methods:

• Utilized cortical vessels as key anatomical features for registration.
• Employed a deep learning approach with feature extraction and pose estimation networks for rigid registration.
• Incorporated a physics-based, non-rigid refinement step to account for brain shift and update tumor location.

Main Results:

• Achieved low pose error in retrospective clinical dataset testing.
• Demonstrated significant brain shift compensation and low Target Registration Error (TRE) on synthetic datasets.
• Validated accuracy below clinical error thresholds, indicating practical feasibility.

Conclusions:

• The developed AR system provides coherent visualization of 3D cortical vessels during surgery.
• Cortical vessels are robust features for both rigid and non-rigid registration in neurosurgery.
• The system shows potential for enhancing surgical precision and patient outcomes.