Generative Adversarial Network for Trimodal Medical Image Fusion Using Primitive Relationship Reasoning

Area of Science:

• Biomedical image processing
• Artificial intelligence in medicine
• Medical imaging analysis

Background:

• Medical image fusion integrates information from multiple imaging modalities for enhanced diagnostics.
• Current research predominantly focuses on dual-modal fusion, leaving a gap in trimodal applications.
• Trimodal medical image fusion offers greater clinical significance and application potential.

Purpose of the Study:

• To propose an end-to-end generative adversarial network (GAN) for trimodal medical image fusion.
• To enhance the fusion process by generating energy maps and utilizing an energy ratio fusion strategy.
• To improve the extraction of global semantic information through attention mechanisms and relationship reasoning.

Main Methods:

• Development of a multi-scale squeeze and excitation reasoning attention network for trimodal fusion.
• Implementation of an energy map generation strategy guided by an energy ratio fusion approach.
• Integration of squeeze and excitation reasoning attention blocks for enhanced global feature extraction and primitive relationship reasoning.

Main Results:

• The proposed method achieved superior visual quality in trimodal medical image fusion compared to existing state-of-the-art techniques.
• Objective evaluation metrics demonstrated the effectiveness of the proposed fusion approach.
• The method achieved the highest accuracy in a subsequent glioma segmentation experiment.

Conclusions:

• The developed generative adversarial network offers an effective solution for trimodal medical image fusion.
• The attention-based approach enhances feature representation and fusion performance.
• The method shows promise for improving both image fusion quality and downstream clinical applications like segmentation.