TPTAF: Task-Prior Tripartite Attention for Infrared and Visible Image Fusion

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Infrared and visible image fusion integrates complementary data for enhanced visual perception and downstream applications.
• Current fusion methods struggle to balance visual quality and detection performance due to competing optimization demands from different tasks.

Purpose of the Study:

• To develop a novel infrared and visible image fusion method that improves both visual quality and downstream task performance.
• To introduce task-prior guidance for better cross-modal feature interaction and optimization.

Main Methods:

• Proposed Task-Prior Tripartite Attention for Infrared and Visible Image Fusion (TPTAF).
• Employed a decoupled encoder to separate infrared and visible features into structural and discriminative spaces.
• Integrated detection semantics as task priors via tripartite attention for guided feature selection.
• Utilized uncertainty-weighted learning to balance fusion and detection losses.

Main Results:

• TPTAF demonstrated stable fusion quality across diverse datasets (M3FD, Road-Scene, AVMS, MSRS).
• The method significantly improved the utility of fused images for object detection and semantic segmentation tasks.
• Achieved a better balance between visual quality and detection performance compared to mainstream methods.

Conclusions:

• TPTAF effectively leverages task-specific semantics to guide the fusion process.
• The proposed method offers a robust solution for infrared and visible image fusion, enhancing performance in critical downstream applications.
• Code availability facilitates further research and application of the TPTAF method.