Elliptical specularity detection in endoscopy with application to normal reconstruction.

Area of Science:

• Computer Vision
• Medical Imaging
• Robotics

Background:

• Specularities in endoscopic imaging are often treated as noise.
• Existing methods for specularity detection produce free-form masks, limiting their utility.
• Reconstructing surface normals from specularities can provide valuable geometric information.

Purpose of the Study:

• To develop a method for detecting specularities as elliptical blobs in endoscopic images.
• To leverage ellipse coefficients for surface normal reconstruction.
• To contrast this approach with previous methods that treat specularities as nuisance.

Main Methods:

• A hybrid pipeline combining deep learning and handcrafted image processing steps.
• Utilizing a fully convolutional network to generate an initial mask of specular pixels.
• Employing standard ellipse fitting for segmentation refinement to identify blobs suitable for normal reconstruction.

Main Results:

• Achieved high detection accuracy with mean Dice scores of 84% (colonoscopy) and 87% (kidney laparoscopy).
• Demonstrated successful surface normal reconstruction with good quantitative agreement with other methods.
• Validated the elliptical shape prior's effectiveness in improving specularity detection.

Conclusions:

• Presents the first fully automatic method for exploiting specularities in endoscopic 3D reconstruction.
• The proposed elliptical specularity detection is simple, generalizable, and potentially valuable for clinical practice.
• Results show promise for future integration with learning-based depth inference and Structure from Motion (SfM) methods.