Self-Supervised Feature Detection and 3D Reconstruction for Real-Time Neuroendoscopic Guidance

Area of Science:

• Neurosurgery
• Medical Imaging
• Computer Vision

Background:

• Transventricular approaches in deep-brain surgery present navigation challenges due to tissue deformation.
• Accurate real-time 3D reconstruction and registration of endoscopic views are crucial for effective neuronavigation.

Purpose of the Study:

• To develop and evaluate a self-supervised feature detection method for enhanced 3D reconstruction and navigation in neuroendoscopy.
• To improve the accuracy of real-time guidance during neurosurgical procedures.

Main Methods:

• A self-supervised learning method (R2D2-E) was trained on unlabeled neuroendoscopic video data from 15 clinical cases.
• The R2D2-E method was integrated into a simultaneous localization and mapping (SLAM) pipeline for 3D reconstruction.
• Performance was evaluated against SIFT, SURF, and SuperPoint for feature matching and 3D reconstruction accuracy.

Main Results:

• R2D2-E demonstrated superior performance in feature matching and 3D reconstruction compared to existing methods.
• R2D2-E features achieved a median projected error of 0.64 mm, outperforming SIFT (0.90 mm), SURF (0.99 mm), and SuperPoint (0.83 mm).
• The method improved F1 score by 14-25% over comparative algorithms.

Conclusions:

• The developed self-supervised feature detection approach enables accurate, real-time 3D reconstruction in neuroendoscopy.
• This method provides robust feature detection despite endoscopic artifacts and accounts for soft-tissue deformation.
• The approach enhances vision-based guidance and augmented visualization, potentially improving neurosurgical accuracy and patient outcomes.