D3R-Net: Dynamic Routing Residue Recurrent Network for Video Rain Removal

Area of Science:

• Computer Vision
• Image Processing
• Artificial Intelligence

Background:

• Video rain removal is crucial for improving visual quality in adverse weather.
• Existing methods struggle with rain occlusion regions where background details are lost.
• Additive rain streak models do not fully capture the complexity of rainy scenes.

Purpose of the Study:

• To develop an advanced method for video rain removal that handles both rain streaks and occlusions.
• To propose a novel hybrid rain model integrating motion segmentation for improved context awareness.
• To introduce the Dynamic Routing Residue Recurrent Network (D3R-Net) for robust video de-raining.

Main Methods:

• A hybrid rain model is proposed to depict both rain streaks and occlusion regions.
• A Dynamic Routing Residue Recurrent Network (D3R-Net) is developed, integrating spatial feature extraction and temporal aggregation.
• Context information, including motion segmentation and rain type indicators, is embedded using a dynamic routing mechanism.

Main Results:

• The D3R-Net effectively extracts spatial features and aggregates them temporally using context-aware recurrent units.
• A context selection gate dynamically selects fused features based on detected context (rain/non-rain regions).
• Extensive experiments demonstrate the superiority of D3R-Net over state-of-the-art methods on synthetic and real-world rainy videos.

Conclusions:

• The proposed D3R-Net achieves superior performance in video rain removal, outperforming existing techniques.
• The hybrid rain model and dynamic routing mechanism effectively address challenges posed by rain occlusions.
• The network design components are validated, confirming their effectiveness in enhancing video quality under rainy conditions.