Learning to See Through Obstructions With Layered Decomposition.

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Image sequences often contain obstructions like reflections, occlusions, and raindrops.
• Removing these obstructions is crucial for various applications, including augmented reality and scene understanding.
• Existing methods struggle with dynamic scenes and imperfect motion estimation.

Purpose of the Study:

• To develop a robust learning-based approach for removing obstructions from image sequences.
• To leverage motion differences for separating background and foreground layers.
• To improve the accuracy and adaptability of obstruction removal techniques.

Main Methods:

• A learning-based method utilizing motion differences between background and obstructions.
• Alternating estimation of dense optical flow fields for two layers (background and obstruction).
• Reconstruction of each layer using a deep convolutional neural network on flow-warped images.

Main Results:

• The method effectively removes various obstructions like reflections and fence occlusions.
• The approach demonstrates good performance on real images, despite being trained on synthetic data.
• The learning-based reconstruction module accommodates errors in flow estimation and relaxes strict assumptions.

Conclusions:

• The proposed method offers an effective solution for obstruction removal in image sequences.
• Leveraging motion cues and deep learning enhances robustness and accuracy.
• The technique shows promise for real-world applications requiring clean image data.