A robust and effective framework for 3D scene reconstruction and high-quality rendering in nasal endoscopy surgery

Area of Science:

• Medical Imaging
• Computer Vision
• Surgical Technology

Background:

• Nasal endoscopic surgery faces challenges due to narrow cavities, limiting surgical view and instrument manipulation.
• Precise real-time intraoperative navigation is vital for avoiding critical anatomical structures in nasal surgeries.
• Existing endoscopic 3D reconstruction methods struggle with nasal scenarios due to data limitations and image quality issues like motion blur and soft tissue deformation.

Purpose of the Study:

• To develop a novel 3D reconstruction model for nasal endoscopy that addresses challenges in data acquisition and image quality.
• To provide precise, real-time 3D information for enhanced intraoperative navigation in nasal endoscopic surgery.
• To improve the visualization of surgical scenes for surgeons, aiding in avoiding critical areas.

Main Methods:

• Collected nasal endoscopy videos with frame-by-frame pose information.
• Proposed Mip-EndoGS, integrating 3D Gaussian Splatting with a diffusion module to reduce motion blur.
• Incorporated an adaptive low-pass filter to mitigate rendering aliasing artifacts.

Main Results:

• Mip-EndoGS successfully reconstructs 3D scenes within the nasal cavity in real-time.
• The model provides surgeons with more detailed and precise surgical scene information.
• Demonstrated significant improvements in handling motion blur and reducing artifacts compared to existing methods.

Conclusions:

• The proposed Mip-EndoGS model offers a robust solution for real-time 3D reconstruction in nasal endoscopy.
• This technology has the potential to significantly enhance intraoperative guidance and surgical safety.
• Future integration with augmented reality (AR)-based surgical navigation systems is promising for advanced surgical assistance.