Weakly-Supervised Depth Estimation and Image Deblurring via Dual-Pixel Sensors.

Area of Science:

• Computer Vision
• Image Processing
• Computational Photography

Background:

• Dual-pixel (DP) sensors capture image pairs per snapshot by splitting pixels, enabling depth estimation.
• Existing DP depth estimation methods struggle with defocus blur, which is inherent to DP disparity.
• Supervised learning for DP depth estimation is hindered by the difficulty of obtaining ground-truth depth maps.

Purpose of the Study:

• To develop a novel method for joint depth estimation and deblurring using DP images.
• To propose a weakly-supervised approach that eliminates the need for ground-truth depth maps during training.
• To improve the performance of depth estimation in DP imaging systems by leveraging blur information.

Main Methods:

• Developed a mathematical model of DP image formation linking blur and depth.
• Proposed the Weakly-supervised Depth and Deblur Network (WDDNet), an extension of DDDNet.
• Implemented an efficient reblur solver and utilized all-in-focus images as supervisory signals for unsupervised depth estimation.

Main Results:

• WDDNet jointly estimates an all-in-focus image and a disparity map.
• The Reblur and Fstack module regularizes disparity estimation and image restoration.
• Experimental results on synthetic and real data show competitive performance against state-of-the-art supervised methods.

Conclusions:

• The proposed WDDNet effectively performs joint depth and deblurring using weakly-supervised learning.
• The method overcomes the limitations of ground-truth data acquisition for DP sensors.
• WDDNet demonstrates robust performance, comparable to supervised approaches, in diverse imaging scenarios.