Evaluation of deep learning-based surface reconstruction and tracking of surgical scenes for minimally invasive intervention

Minimally invasive procedures enable clinicians to perform interventions with reduced tissue damage and expedited recovery times. However, laparoscopic imaging inherently provides a limited field of view, compelling surgeons to mentally integrate preoperative imaging with intraoperative scenes, which is prone to localization errors. Augmented reality (AR) offers a solution by overlaying preoperative 3D organ models onto live laparoscopic video; however, precise 3D reconstruction and localization of the surgical scene is critical for accurate overlays. In this study, we evaluated three deep learning-based surface reconstruction methods, DROID-SLAM, MASt3R-SLAM, and the visual geometry grounded transformer (VGGT), for reconstructing tissue surfaces from laparoscopic video. We evaluated these algorithms with a dataset comprised of laparoscopic RGB frames of ex-vivo porcine tissue, ground-truth camera trajectories obtained via an optical tracking system and known camera intrinsic parameters. Dense point cloud and depth map predications were then qualitatively compared between models and estimated camera trajectory was compared quantitively using absolute trajectory error (ATE) and relative pose error (RPE). Preliminary results indicate that all models produced plausible tissue surface reconstructions, with estimated depth maps and camera trajectories demonstrating close alignment with the ground truth. These findings suggest that deep learning-based 3D reconstruction methods can effectively support image-guided interventions without relying on ground-truth camera poses or depth maps and may be integrated with AR to support minimally invasive surgical workflows.