MRCE-Net: A multi-role collaborative experts deep learning network for multi-modal medical image fusion

Multi-modal medical image fusion aimed to combine images from different modalities to leverage their complementary strengths and mitigate the limitations of individual imaging techniques. In recent years, deep learning-based approaches became the dominant direction, surpassing traditional methods in this field. However, existing medical image fusion methods struggle to balance local feature extraction with global context representation, and to effectively capture the specificity and complementarity of different modalities. To overcome these limitations, we propose a Multi-Role Collaborative Experts Network, termed MRCE-Net, for multi-modal medical image fusion. Specifically, we employed a dual-branch encoder to extract modality-specific features from each modality. This encoder integrated a window-based Transformer for local feature extraction and a global channel-based Transformer for capturing long-range contextual dependencies, effectively balancing both aspects. In addition, we propose a Multi-Role Collaborative Experts fusion module that enables specialized experts to jointly model distinct aspects of multi-modal features, with a particular focus on capturing both modality-specific characteristics and inter-modality complementarity. By exploiting the synergistic capabilities of the experts, our framework achieved more comprehensive feature representation and more accurate fusion results. Extensive experiments on a public multi-modal medical image fusion benchmark and an in-house collected brain anatomical and functional imaging dataset demonstrate that our method outperforms state-of-the-art approaches in both visual quality and quantitative performance. The source code will be made publicly available upon publication at https://github.com/Dpw506/MRCENet.