Related Experiment Video
Updated: May 14, 2026
06:54
Photorealistic Learned Landscapes for Augmented Reality
Published on: June 27, 2025
LumenGSLAM: online physically based rendering with Gaussian Splatting for robust endoscopic reconstruction and
Francesco Leni1,2, Chiara Lena3, Zhehua Mao4
1Department of Electronics, Information, and Bioengineering, Politecnico di Milano, Via Giuseppe Ponzio, 34, Milan, 20133, Italy. francesco.leni@mail.polimi.it.
Summary
This study introduces LumenGSLAM, an online framework for realistic 3D endoscopic reconstruction and tracking. It achieves stable, photorealistic rendering and accurate anatomical mapping for improved intraoperative navigation.
Area of Science:
- Computer Vision
- Medical Imaging
- Robotics
Background:
- Realistic 3D reconstruction from endoscopic video is crucial for intraoperative procedures.
- Existing methods struggle with accurate light modeling, offline processing, and unstable tracking.
Purpose of the Study:
- To develop an online framework for stable tracking and photorealistic endoscopic rendering.
- To overcome limitations of existing approaches in realistic light modeling and offline optimization.
Main Methods:
- Proposed LumenGSLAM, an online RGB-D Gaussian Splatting framework for dense endoscopic reconstruction.
- Utilized dense depth input for stable geometry estimation and physically based rendering (PBR).
- Implemented surface-aligned Gaussian initialization and gradient scaling for anatomical fidelity and robust camera pose estimation via feature-based tracking.
Main Results:
- LumenGSLAM demonstrated superior online reconstruction and tracking on C3VD and SCARED datasets.
- Achieved high PSNR (30.6), SSIM (0.89), and low LPIPS (0.23) on C3VD, surpassing online baselines.
- Exhibited lowest Absolute Trajectory Error (ATE = 0.93 mm) and Rotational Error (ARE = 0.98°), proving robustness in challenging conditions.
Conclusions:
- LumenGSLAM sets a new benchmark for online RGB-D endoscopic reconstruction.
- Achieved photometrically consistent and anatomically accurate mapping using explicit light modeling and Gaussian optimization.
- Its robustness offers potential for intraoperative navigation and dynamic tissue modeling.
