EndoLRMGS: Combining Large Reconstruction Modelling and Gaussian Splatting for Complete Endoscopic Scene...

Reconstructing dynamic surgical scenes from endoscopic videos remains a fundamental challenge in robot-assisted surgery. Existing methods primarily focus on deformable tissues, overlooking the presence of articulated instruments. To bridge this gap, we present EndoLRMGS, the first unified framework capable of reconstructing both deformable tissue and articulated instruments in a modular approach from monocular video and depth priors. We introduce Frequency-Modulated Gaussian Splatting (FMGS), for deformable tissue reconstruction, which modulates the spatial frequency of Gaussian primitives according to the Nyquist-Shannon sampling theorem, improving the visual fidelity while maintaining robust geometric accuracy. For instrument reconstruction, we leverage the Large Reconstruction Model (LRM) to generate high-quality, watertight 3D models from single images, and introduce a novel Orthographic and Perspective joint Projection Optimization (OPjPO) module to recover metric scale and spatial alignment. Extensive experiments on public datasets demonstrate the effectiveness of EndoLRMGS, achieving PSNR values from 28.4981 to 38.4179, with Chamfer distance ranging from 1.43 to 4.71 mm in tissue reconstruction. For instrument reconstruction, EndoLRMGS achieves PSNR values ranging from 19.7341 to 22.4393 on left views and 17.7442 to 20.8436 on right views. In terms of spatial alignment accuracy, it attains IoU values between 71.56% and 85.82%, with Chamfer distance ranging from 12.59 to 17.38 mm. These results highlight EndoLRMGS as a powerful and versatile solution for accurate, complete, and photorealistic 3D reconstruction of surgical scenes. Code is available at: EndoLRMGS.