4D monocular surgical reconstruction under arbitrary camera motions.

Area of Science:

• Computer Vision
• Medical Imaging
• Surgical Robotics

Background:

• Reconstructing deformable surgical scenes from endoscopic videos is crucial for clinical applications.
• Current methods struggle with monocular sequences and significant camera motion due to reliance on stereo depth or accurate structure-from-motion.
• Existing approaches are limited to fixed endoscope viewpoints, restricting real-world clinical utility.

Purpose of the Study:

• To develop a high-quality 4D reconstruction framework for monocular endoscopic sequences with arbitrary camera motion.
• To address limitations of existing methods in handling large camera movements and lack of stereo depth priors.
• To enable scalable and robust reconstruction of deformable surgical scenes.

Main Methods:

• Proposed Local-EndoGS, a progressive, window-based global scene representation framework.
• Introduced a coarse-to-fine initialization strategy integrating multi-view geometry, cross-window information, and monocular depth priors.
• Incorporated long-range 2D pixel trajectory constraints and physical motion priors for enhanced deformation plausibility.

Main Results:

• Local-EndoGS achieved superior performance in appearance quality and geometric accuracy on deformable endoscopic datasets.
• Consistently outperformed state-of-the-art methods in reconstructing scenes with varying camera motions.
• Ablation studies validated the effectiveness of the proposed framework's key components.

Conclusions:

• Local-EndoGS offers a robust and scalable solution for 4D reconstruction of deformable surgical scenes from monocular endoscopic videos.
• The framework effectively handles arbitrary camera motion and overcomes initialization challenges.
• This advancement holds significant potential for improving surgical planning and navigation.