Physics-Based Shadow Image Decomposition for Shadow Removal.

Area of Science:

• Computer Vision
• Deep Learning
• Image Processing

Background:

• Shadows in images degrade visual quality and hinder downstream tasks.
• Existing shadow removal methods often require paired shadow and shadow-free images, which are difficult to obtain.

Purpose of the Study:

• To propose a novel deep learning framework for effective shadow removal.
• To develop a weakly-supervised shadow removal method that does not require paired data.
• To introduce a new dataset for video shadow removal evaluation.

Main Methods:

• A linear illumination transformation models shadow effects, decomposing images into shadow-free components, shadow parameters, and a matte layer.
• Two deep networks, SP-Net and M-Net, predict shadow parameters and matte, respectively.
• An inpainting network (I-Net) refines the shadow-free image, and a patch-based weakly-supervised approach is formulated.

Main Results:

• The proposed method achieves a 20% improvement in mean absolute error (MAE) on the ISTD dataset.
• The weakly-supervised model yields competitive results without paired shadow-free images.
• A new dataset, SBU-Timelapse, is introduced for video shadow removal.

Conclusions:

• The novel deep learning framework effectively removes shadows and enables weakly-supervised training.
• The method advances the state-of-the-art in shadow removal and offers a more practical approach.
• The new dataset will facilitate further research in video shadow removal.