Heavy Rain Face Image Restoration: Integrating Physical Degradation Model and Facial Component-Guided Adversarial...

Area of Science:

• Computer Vision
• Image Processing
• Artificial Intelligence

Background:

• Intelligent CCTVs require advanced image processing for effective visual surveillance.
• Heavy rain significantly degrades face image visibility and resolution in CCTV footage.
• Existing image degradation models do not address the combined challenge of resolution loss and heavy rain.

Purpose of the Study:

• To propose a novel scale-aware heavy rain model for image degradation.
• To develop a method for restoring high-resolution face images (HR-FIs) from low-resolution heavy rain face images (LRHR-FIs).
• To simultaneously remove heavy rain, increase resolution, and improve visibility of face images.

Main Methods:

• A two-stage network is presented for image restoration.
• The first stage uses an interpretable image degradation model (IDM)-based network to predict physical parameters and remove rain.
• The second stage employs facial component-guided adversarial learning (FCGAL) with a face parsing-guided generator and local discriminators for enhanced facial structure restoration.

Main Results:

• The proposed method successfully removes heavy rain and enhances resolution and visibility.
• Experimental results demonstrate superior performance compared to state-of-the-art models in heavy rain removal, image-to-image translation, and super-resolution.
• The interpretable IDM-based network improves physical parameter estimation through image reconstruction loss.

Conclusions:

• The developed approach effectively restores high-resolution face images from degraded low-resolution heavy rain inputs.
• The combination of a physical-based network and FCGAL offers a robust solution for challenging surveillance scenarios.
• This work advances the field of image restoration for real-world applications like CCTV surveillance.