RTFusion: A Multimodal Fusion Network with Significant Information Enhancement

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Multimodal medical fusion images are vital for accurate clinical diagnosis, offering detailed anatomical and disease location information.
• Existing fusion methods often result in significant information loss, compromising diagnostic quality.

Purpose of the Study:

• To design a novel multimodal fusion network, RTFusion, that enhances information preservation and visual quality in medical image fusion.
• To address the limitations of current methods by minimizing information loss and improving the subjective and objective evaluation of fused images.

Main Methods:

• Developed a Residual Transformer Fusion Network (RTFusion) incorporating remote image information interaction for global context and residual structures for feature enhancement.
• Integrated Channel Attention and Spatial Attention Module (CASAM) to boost significant information and a feature interaction module for source-specific information exchange.
• Designed a block calculation loss function to preserve rich texture, structural, and color details, optimizing visual effects.

Main Results:

• RTFusion demonstrated superior performance in recovering significant source image information compared to advanced methods.
• The proposed method achieved better subjective visual descriptions and objective metric evaluations.
• Balanced enhancement of color and texture information improved the overall visual quality of the fused images.

Conclusions:

• RTFusion effectively enhances multimodal medical image fusion by minimizing information loss and preserving critical details.
• The network offers improved diagnostic potential through superior image quality and information recovery.
• This approach represents a significant advancement in medical image fusion technology for clinical applications.