Vision-Guided Surgical Navigation Using Computer Vision for Dynamic Intraoperative Imaging Updates

Area of Science:

• Medical Imaging
• Computer Vision
• Surgical Technology

Background:

• Residual disease after endoscopic sinus surgery (ESS) leads to poor outcomes and repeat surgeries.
• Current image-guided surgery systems lack dynamic intraoperative updates.
• A sensorless, video-based method using neural radiance fields (NeRF) for intraoperative CT updating is proposed.

Purpose of the Study:

• To develop and validate a novel method for dynamic intraoperative CT updating using NeRF.
• To assess the accuracy of NeRF-generated 3D surgical field reconstructions compared to preoperative CT scans.
• To evaluate the potential of this technology to improve surgical completeness in ESS.

Main Methods:

• Bilateral ESS was performed on 3D-printed models.
• Endoscopic videos were processed using a custom NeRF pipeline for 3D reconstruction.
• NeRF reconstructions were co-registered to preoperative CT scans, and updated CT models were generated and compared to ground-truth postoperative CT using Hausdorff distance, DSC, and BAA.

Main Results:

• Sub-millimeter surface alignment was achieved with an average Hausdorff distance of 0.27 ± 0.076 mm.
• Excellent volumetric overlap was demonstrated with a Dice Similarity Coefficient (DSC) of 0.93 ± 0.012.
• Bland-Altman analysis showed acceptable volume agreement, with a mean difference of 0.40 cm³.

Conclusions:

• Computer vision, specifically NeRF, enables accurate, dynamic intraoperative CT updates from endoscopic video without external tracking.
• This software-driven tool visualizes resection progress, potentially enhancing surgical completeness in ESS.
• The technology offers a pathway for next-generation surgical navigation platforms.