DAMFusion: An end-to-end degradation-aware multi-modal image fusion framework

Multi-modal image fusion aims to integrate complementary information from different modalities into a more comprehensive and robust unified representation, thereby better supporting downstream tasks such as object detection and image recognition. However, most existing methods often lack explicit modeling capabilities for degraded features when faced with degraded images, or struggle to achieve fine-grained adaptive processing for diverse compound degradation scenarios. This results in unstable fusion performance under complex environments. To address these challenges, this paper proposes an end-to-end degradation-aware multi-modal image fusion framework (DAMFusion). The framework is built upon a two-stage autoencoder backbone, incorporating the sparse routing mechanism of a Mixture of Experts (MoE) along with heterogeneous expert structures. Guided by degradation semantics, the algorithm enables on-demand activation and task-specific optimization of expert modules, effectively balancing the trade-off between shared representation and task specificity. Enhanced fusion robustness and efficient utilization of expert modules are further achieved by introducing a mutual information based sparse constraint with KL divergence. This constraint mitigates the routing collapse problem, promotes functional diversity and load balancing among experts, and reduces redundancy and interference across expert modules, thereby strengthening the overall model stability. The proposed method demonstrates state-of-the-art (SOTA) or comparable performance on infrared-visible image fusion tasks as well as medical multi-modal image fusion tasks. In addition, cross-dataset evaluations reveal strong stability and generalization capability, highlighting its substantial practical value. The implementation is available at: DAMFusion(GitHub).