Semantic-Aware Fusion Network Based on Super-Resolution.

Area of Science:

• Computer Vision
• Image Processing
• Artificial Intelligence

Background:

• Existing infrared and visible image fusion methods often produce low-resolution images with lost details due to hardware limitations.
• Current fusion algorithms prioritize visual quality over the requirements of high-level vision tasks, hindering practical applications.

Purpose of the Study:

• To develop a novel fusion network that addresses the limitations of low-resolution images and enhances performance in high-level vision tasks.
• To integrate super-resolution, fusion, and segmentation networks for improved semantic understanding and visual quality in fused images.

Main Methods:

• A super-resolution network utilizing a multi-branch hybrid attention module (MHAM) was designed to enhance source image quality and details.
• A comprehensive information extraction module (STDC) was incorporated into the fusion network to capture finer-grained complementary information.
• Joint training of the fusion and segmentation networks with semantic loss was employed to guide semantic information back into the fusion process.

Main Results:

• The proposed super-resolution-based semantic-aware fusion network effectively enhances the quality and details of source images.
• The method demonstrates superior performance compared to state-of-the-art image fusion techniques in extensive experiments.
• Fused images exhibit excellent visual perception and significantly improve the performance of high-level vision tasks.

Conclusions:

• The developed network successfully overcomes challenges associated with low-resolution images in fusion tasks.
• The semantic-aware approach ensures that fused images are not only visually appealing but also semantically rich for downstream applications.
• This work offers a promising direction for advancing infrared and visible image fusion for both visual quality and task-specific performance.