Joint Reflection Removal and Depth Estimation From a Single Image

Reflection caused by glass often degrades the quality of an image and further makes it difficult to estimate depth. In this article, we propose joint reflection removal and depth estimation from a single image. We perform reflection removal (transmission recovery) and depth estimation jointly using a collaborative neural network that consists of four blocks: 1) encoder for feature extraction; 2) reflection removal subnetwork (RRN); 3) depth estimation subnetwork (DEN); and 4) depth refinement guided by the transmission layer. We achieve collaboration between reflection removal and depth estimation by concatenating intermediate features of DEN with RRN. Since the recovered transmission layer contains accurate edges of objects behind glass, we refine the estimated depth with its guidance by guided image filtering. The experimental results demonstrate that the proposed method achieves both reflection removal and depth estimation even for images with dominant reflections. Besides, this article offers a new way of treating reflections in images to introduce depth estimation into reflection removal and achieve reflection removal and depth estimation simultaneously.