Multi-scene image fusion via memory aware synapses.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Existing image fusion methods struggle with multi-scene environments and low-light conditions.
• Visible images degrade severely in low-light, causing poor feature representation and fusion quality.
• Task switching in multi-scene fusion leads to knowledge loss and color information damage.

Purpose of the Study:

• To propose a multi-scene image fusion method (MMF-Fusion) that improves fusion quality under varying conditions.
• To enhance feature representation and mitigate knowledge loss during task switching.
• To provide effective information for high-level vision tasks through multimodal fusion and low-illumination enhancement.

Main Methods:

• A hybrid CNN-Transformer structure fuses local and global features for enhanced scene representation.
• A novel FFM structure integrates multi-scale and multi-scene features for improved fusion.
• Memory Aware Synapses (MAS) continuous learning trains the model, preserving visible light features and reducing color damage.

Main Results:

• MMF-Fusion demonstrates superior visual quality and quantitative evaluation compared to state-of-the-art algorithms.
• The method effectively enhances fused images in low-illumination scenarios.
• Improved multimodal fusion provides richer information for subsequent vision tasks.

Conclusions:

• MMF-Fusion offers a robust solution for multi-scene image fusion, particularly under challenging low-light conditions.
• The continuous learning approach successfully preserves crucial image features and mitigates knowledge loss.
• The proposed method significantly advances the field of image fusion for practical applications.