Neural Radiance Fields for Novel View Synthesis in Monocular Gastroscopy.

Area of Science:

• Medical Imaging
• Computer Vision
• Computational Geometry

Background:

• Traditional 3D reconstruction for novel viewpoint synthesis in gastroscopy often yields incomplete or noisy results due to low-texture and non-Lambertian stomach regions.
• Existing methods struggle with view sparsity, limiting the quality of rendered images for diagnostic purposes.

Purpose of the Study:

• To apply neural radiance fields (NeRF) for synthesizing high-fidelity, photo-realistic images from novel viewpoints within the stomach using monocular gastroscopic data.
• To improve NeRF performance in sparse monocular gastroscopy by integrating geometry priors from pre-reconstructed point clouds.

Main Methods:

• Utilized neural radiance fields (NeRF) for novel viewpoint image synthesis from monocular endoscopic images.
• Incorporated geometry priors from pre-reconstructed point clouds into NeRF training.
• Introduced a novel geometry-based loss function applied to both observed and generated views.

Main Results:

• Achieved high-fidelity image renderings from novel viewpoints within the stomach.
• Demonstrated superior qualitative and quantitative performance compared to other recent NeRF methods.
• Successfully addressed performance degradation caused by view sparsity in monocular gastroscopy.

Conclusions:

• The proposed NeRF-based approach with geometry priors effectively synthesizes realistic novel viewpoint images from monocular gastroscopic data.
• This method overcomes limitations of traditional 3D reconstruction techniques in challenging endoscopic environments.
• Offers a promising advancement for stomach diagnosis through enhanced 3D visualization and image rendering.