C²LNet: Fusing infrared and visible image via a cross-modality complementarity learning network

Mining and fusing complementary information across modalities are crucial for infrared and visible image fusion (IVIF), yet existing methods largely rely on implicit feature aggregation and fail to explicitly mine and preserve cross-modality complementarity. To handle this issue, this paper proposes a Cross-modality Complementarity Learning Network (C²LNet) that simultaneously mines complementary features from different modalities and ensures the embedding of those mined cues in fused images. Specifically, C²LNet uses an Interactive Complementarity Mining (ICM) method to guide the network to mine and extract complementary features from infrared and visible images with respect to different feature levels and spatial regions, providing more diverse and informative features for IVIF. To encourage fused images to contain visual features from different modalities and to be more discriminative for downstream semantic tasks, a Complementarity Fusing Constraint (CFC) is designed to supervise the training of C²LNet. CFC ensures similar embeddings of fused images and input images, and additionally enhances the discrimination capacity of fused images, thus is more effective in supervising the fusion of those complementary features mined by ICM. Extensive experiments on three widely-used datasets demonstrate that our method outperforms state-of-the-art methods by clear margins, e.g., improving VIF metric by 3.2% on RoadScene, 6.0% on M³FD, and 4.6% on LLVIP, respectively. Further experiments in an electricity factory show that C²LNet also generalizes to real-world industrial scenarios, showing potential to benefit engineering applications such as autonomous robotic inspection. Our code is available at https://github.com/XingyuNKU/C2LNet.