Related Experiment Videos
Visible-Infrared Fusion Based on CNN and Deformable Transformer
Xiaoyi Wang1,2, Xiansong Gu1, Bin Li2
1College of Opto-Electronic Engineering, Changchun University of Science and Technology, Changchun 130022, China.
Journal of Imaging
|June 25, 2026
Summary
This study introduces a novel infrared-visible image object detection method using Convolutional Neural Networks (CNNs) and Deformable Transformers. The approach enhances object detection in challenging conditions like low light and occlusion.
Area of Science:
- Computer Vision
- Artificial Intelligence
- Image Processing
Background:
- Traditional feature extraction and multi-modal fusion methods have limitations.
- Integrating infrared and visible imagery presents challenges in fusing local and global information.
Purpose of the Study:
- To propose an advanced object detection architecture for infrared-visible images.
- To improve multi-modal information fusion and feature extraction for enhanced detection accuracy.
Main Methods:
- Integration of Convolutional Neural Networks (CNNs) for local feature modeling.
- Utilization of Deformable Transformers for global contextual information capture.
- Implementation of a detection-aware multi-task optimization mechanism.
Main Results:
- Achieved competitive or superior performance on M3FD and LLVIP datasets.
- Obtained the highest mAP50 scores: 74.2% on M3FD and 98.6% on LLVIP.
- Demonstrated significant improvements over existing methods, including PIAFusion.
Conclusions:
- The proposed CNN-Transformer architecture effectively fuses multi-modal image data.
- The method shows practical effectiveness and robustness in complex environments with low light and occlusion.
