A Dataset for Depth in Robotic Endoscopy with Dynamic Scenarios (DRENDS)

Depth perception in robotic minimally invasive surgery remains a critical challenge for many downstream tasks, demanding advanced depth estimation techniques and ground truth data for their validation. Current datasets lack data with ground-truth depth information in dynamic scenarios; therefore, we present DRENDS (Depth in Robotic Endoscopy with Dynamic Scenarios)¹, a novel dataset comprising sequences of high-resolution stereo images captured during the robotic laparoscopic manipulation of a human phantom and ex vivo porcine tissue, along with ground-truth point clouds for each frame and calibration data. The data were collected under three illumination conditions and across different anatomies involving tissue manipulation and non-rigid deformations. Our code for rectifying stereo images, handling camera-perspective occlusions, and obtaining depth maps per frame is open source for reproducibility and easy adaptation. Finally, we also conduct baseline evaluations using state-of-the-art depth estimation models to establish benchmark performance on our dataset. The results and data highlight the challenges and potential of metric temporally consistent depth estimation in robotic surgery, encouraging further advancements in tissue deformation prediction for medical applications. We publicly release DRENDS¹ to foster innovation and collaboration in this critical field.