Kidney Endoscopy Video to Preoperative CT Alignment for Depth Estimation

Area of Science:

• Medical Imaging
• Computer Vision
• Surgical Robotics

Background:

• Accurate depth perception is crucial for kidney endoscopy, but obtaining ground truth depth for training neural networks is difficult.
• Traditional methods like optical tracking are incompatible with kidney endoscopy, hindering direct depth information acquisition.

Purpose of the Study:

• To develop a robust method for generating accurate ground truth depth information for kidney endoscopy.
• To enable effective training of neural networks for monocular depth estimation in this surgical context.

Main Methods:

• A pipeline was developed to reconstruct a 3D point cloud from endoscopy videos.
• This point cloud was registered to a segmentation derived from preoperative CT scans.
• Synthetic endoscopy frames with precise depth information were generated using estimated camera poses and CT surface geometry.

Main Results:

• The method achieved sub-millimeter Root Mean Square Error (RMSE) accuracy on kidney phantoms (0.61 ± 0.08 mm).
• Validation on real patient data demonstrated comparable accuracy (0.86 mm).

Conclusions:

• The proposed pipeline effectively generates accurate ground truth depth for kidney endoscopy.
• This approach overcomes limitations of real-time tracking and facilitates neural network training for improved surgical visualization.