Toward Better De-Raining Generalization via Rainy Characteristics Memorization and Replay.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Current image de-raining methods struggle with real-world variability due to limited training data.
• Existing approaches lack adaptability to diverse and evolving rainy conditions.

Purpose of the Study:

• To develop a novel framework for progressively expanding image de-raining networks' knowledge base.
• To enhance the adaptability and generalization capabilities of de-raining models using a complementary learning system inspired approach.

Main Methods:

• Utilized generative adversarial networks (GANs) to capture and retain features from new datasets, mimicking hippocampal memory functions.
• Implemented a training strategy involving existing and GAN-synthesized data, mirroring hippocampal replay and interleaved learning.
• Employed knowledge distillation with replayed data to integrate new knowledge with existing neocortical knowledge.

Main Results:

• The proposed framework demonstrated effective continual knowledge accumulation across six datasets.
• Tested on three benchmark de-raining networks, the framework significantly improved performance on previously unseen rainy scenes.
• The approach surpassed state-of-the-art methods in generalizing to new real-world de-raining scenarios.

Conclusions:

• The complementary learning system framework enables robust knowledge accumulation for image de-raining.
• This method significantly enhances the adaptability and generalization of de-raining networks to diverse real-world conditions.
• The findings offer a promising direction for developing more resilient and effective image de-raining solutions.