Calibration-free structured-light-based 3D scanning system in laparoscope for robotic surgery

Area of Science:

• Medical Robotics
• Computer Vision
• Surgical Technology

Background:

• Accurate 3D shape measurement is vital for robotic surgery systems, particularly for surgical support and alignment.
• Stereo cameras in laparoscopes face accuracy limitations in low-texture environments, hindering precise 3D reconstruction.
• Existing deep learning methods for stereo matching require extensive depth-annotated datasets, which are scarce for laparoscopic applications.

Purpose of the Study:

• To develop an effective 3D shape reconstruction method for laparoscopic surgery that overcomes the limitations of texture-less environments.
• To enable accurate depth reconstruction without relying on large, annotated datasets.

Main Methods:

• Proposed a 3D shape reconstruction method utilizing an ultra-small patterned projector integrated with a laparoscopic arm.
• Employed structured light with a grid-like pattern for positional encoding.
• Implemented auto-calibration of projector-camera relative poses via differential rendering and captured multiple images during projector motion.

Main Results:

• Successfully performed 3D reconstruction using images acquired from a surgical robot.
• Evaluated the method's accuracy by comparing reconstructed shapes with ground-truth data from X-ray CT scans.
• Demonstrated the feasibility of active stereo techniques for 3D reconstruction in challenging laparoscopic scenarios.

Conclusions:

• The proposed patterned projector method offers a viable solution for accurate 3D shape measurement in laparoscopy, especially in texture-deficient conditions.
• This approach addresses the need for depth reconstruction without extensive annotated datasets, paving the way for improved robotic surgery support.
• Active stereo techniques with structured light show significant promise for advancing 3D reconstruction in minimally invasive surgery.