Dense surface reconstruction with shadows in MIS.

Area of Science:

• Medical imaging
• Computer vision
• Surgical robotics

Background:

• Accurate 3D reconstruction of internal organs is crucial for guiding surgical tools in minimally invasive surgery (MIS).
• Existing stereo camera techniques struggle with limited texture and specular reflections on organ surfaces, hindering precise correspondence matching.
• This limits the effectiveness of current 3D reconstruction methods in MIS environments.

Purpose of the Study:

• To develop a novel 3D reconstruction approach for MIS that overcomes limitations of texture-based stereo methods.
• To improve the accuracy and efficiency of 3D surface reconstruction by utilizing surgical tool shadows.
• To enable better control and guidance of surgical tools during MIS procedures.

Main Methods:

• Proposed a method using weakly structured light to cast surgical tool shadows onto organ surfaces.
• Developed a robust algorithm for extracting shadow edges from image sequences.
• Introduced a novel field surface interpolation (FSI) technique to generate accurate and dense disparity maps.
• Leveraged shadow information, eliminating the need for explicit stereo matching and relying minimally on texture.

Main Results:

• Achieved high precision in 3D surface reconstruction: 0.7 mm for phantom models and 1.2 mm for ex vivo images.
• Demonstrated superior performance compared to state-of-the-art stereo algorithms for MIS when shadows were incorporated.
• Validated the method's effectiveness using phantom models and ex vivo surgical images.

Conclusions:

• The proposed shadow-based 3D reconstruction method significantly enhances accuracy and efficiency in MIS.
• This technique offers a robust solution for 3D surface reconstruction in challenging surgical environments with limited texture.
• The minimal hardware requirements and direct point correspondence calculation make it a practical and efficient solution for surgical guidance.