AWM-Fuse: Multi-Modality Image Fusion for Adverse Weather via Global and Local Text Perception.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Adverse weather degrades visual information in multi-modality image fusion (MMIF).
• Existing methods struggle with effective textual information integration for semantic perception.
• Lack of comprehensive analysis and categorization of textual content in prior work.

Purpose of the Study:

• To propose AWM-Fuse, a unified framework for MMIF under diverse weather conditions.
• To enhance semantic perception and visual detail recovery using global and local text perception.
• To introduce AWMM-Text, a large-scale benchmark for text-guided fusion in adverse weather.

Main Methods:

• AWM-Fuse utilizes a global feature perception module with BLIP-generated captions for overall scene understanding and degradation type identification.
• A local module employs ChatGPT-generated descriptions for detailed analysis of specific degradation effects and recovery of subtle details.
• Textual descriptions constrain fusion image generation, aligning network learning with semantic labels for improved feature learning.

Main Results:

• AWM-Fuse demonstrates superior performance compared to state-of-the-art methods in complex weather conditions.
• The framework shows consistent improvements across multiple downstream tasks.
• The proposed AWMM-Text benchmark facilitates effective text-guided fusion under adverse weather.

Conclusions:

• AWM-Fuse offers a robust solution for MMIF in adverse weather by integrating global and local text perception.
• The framework generalizes well across various weather conditions and improves semantic understanding.
• The study provides a valuable benchmark and method for advancing text-guided image fusion research.