Multi-sensor super-resolution for hybrid range imaging with application to 3-D endoscopy and open surgery

Area of Science:

• Computer Vision
• Medical Imaging
• Image Processing

Background:

• Hybrid 3-D imaging combines different data modalities for enhanced scene understanding.
• Super-resolution techniques aim to increase image resolution from low-resolution inputs.
• Image-guided surgery requires high-resolution data for accurate navigation and intervention.

Purpose of the Study:

• To develop a multi-sensor super-resolution framework for enhancing low-resolution range images using complementary photometric data.
• To improve the reconstruction of depth discontinuities in hybrid 3-D range imaging.
• To leverage advancements in image-guided surgery through enhanced data quality.

Main Methods:

• A maximum a-posteriori (MAP) based super-resolution formulation utilizing multiple low-resolution range frames and photometric guidance.
• Robust subpixel motion estimation on photometric data to guide range image super-resolution.
• Novel adaptive regularization exploiting cross-modality correlations for improved depth discontinuity reconstruction.

Main Results:

• The proposed framework demonstrated an average improvement of 2 dB in peak signal-to-noise ratio (PSNR) and 0.03 in structural similarity (SSIM) compared to single-sensor methods.
• Significant enhancements in the reconstruction of depth discontinuities were observed in ex-vivo experiments on porcine organs.
• Successful evaluation on both synthetic and real-world ex-vivo surgical data (open surgery and endoscopy).

Conclusions:

• The multi-sensor super-resolution approach effectively enhances hybrid 3-D range imaging by integrating photometric guidance.
• The method offers substantial improvements in image fidelity and the accurate representation of scene geometry, particularly at depth discontinuities.
• This framework holds promise for advancing image-guided surgical procedures through superior data visualization and analysis.