MIS-NeRF: neural radiance fields in minimally-invasive surgery.

Area of Science:

• Medical Imaging
• Computer Vision
• Surgical Technology

Background:

• Minimally-invasive surgery (MIS) offers reduced trauma but poses challenges for accurate lesion localization.
• Augmented reality (AR) systems aim to improve visualization by overlaying preoperative 3D models onto intraoperative MIS images.
• Precise registration of 3D models to MIS images is a critical, yet difficult, step for effective AR assistance.

Purpose of the Study:

• To develop and evaluate MIS-NeRF, a novel neural radiance field (NeRF) approach for high-fidelity intraoperative 3D reconstruction in MIS.
• To utilize MIS-NeRF to bootstrap and improve the accuracy of 3D model to MIS image registration for lesion localization.
• To adapt NeRF methods to the specific challenges of the MIS environment, such as moving light sources and specular highlights.

Main Methods:

• Proposed MIS-NeRF incorporates camera center input to handle dynamic lighting conditions inherent in MIS.
• A modified volume rendering technique was developed to effectively manage specular highlights during reconstruction.
• A regularized compound loss function was employed to enhance the fidelity of surface reconstruction.

Main Results:

• MIS-NeRF successfully reconstructed high-fidelity liver and uterus surfaces from both synthetic data and retrospective laparoscopic surgeries.
• The method significantly reduced common artifacts like high-frequency noise and bumps caused by specular reflections.
• Iterative Closest Point (ICP) registration using MIS-NeRF achieved an average alignment error of 3.25 mm, outperforming existing methods by 15%.

Conclusions:

• MIS-NeRF provides a robust solution for high-fidelity intraoperative 3D reconstruction in MIS.
• The improved registration accuracy facilitated by MIS-NeRF enhances AR-based lesion localization capabilities.
• This advancement holds promise for improving surgical guidance and outcomes in MIS procedures.