ShadowGAN-Former: Reweighting self-attention based on mask for shadow removal

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Shadow removal is crucial for image restoration but challenging due to inconsistencies in existing methods.
• Many approaches neglect valuable information from non-shadow regions, impacting image quality.

Purpose of the Study:

• To develop an efficient hybrid model for improved shadow removal.
• To leverage non-shadow region information for more consistent and accurate shadow-free image reconstruction.

Main Methods:

• Proposed ShadowGAN-Former, a hybrid Transformer and Generative Adversarial Network (GAN).
• Introduced Multi-Head Transposed Attention (MHTA) and Gated Feed-Forward Network (Gated FFN) for efficient feature extraction.
• Developed the Shadow Attention Reweight Module (SARM) to reweight attention maps based on shadow-non-shadow region correlation.

Main Results:

• ShadowGAN-Former demonstrated superior performance over state-of-the-art methods on ISTD and SRD datasets.
• The SARM module significantly improved Peak Signal-to-Noise Ratio (PSNR) by 5.42%.
• The SARM module reduced Root Mean Square Error (RMSE) by 14.76%.

Conclusions:

• ShadowGAN-Former offers an effective solution for shadow removal by incorporating contextual information.
• The proposed attention mechanisms and modules enhance the model's ability to restore shadow-affected image regions accurately.