Medical image fusion quality assessment based on conditional generative adversarial network

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Multimodal medical image fusion (MMIF) enhances disease diagnosis and treatment.
• Existing MMIF methods lack dedicated quality assessment strategies.
• Objective evaluation is crucial for reliable MMIF.

Purpose of the Study:

• To propose a novel quality assessment method for MMIF.
• To leverage conditional generative adversarial networks (cGANs) for MMIF quality evaluation.
• To ensure the reliability and accuracy of MMIF in clinical applications.

Main Methods:

• Developed a cGAN-based framework for MMIF quality assessment.
• Employed a dual-channel encoder-decoder to extract features guided by Mean Opinion Scores (MOS).
• Utilized a self-attention feature block for hierarchical feature fusion and a discriminator to refine the generator's objective.

Main Results:

• The proposed method achieved state-of-the-art performance on an established MMIF database.
• Demonstrated excellent agreement between the automated assessment and subjective human evaluations.
• The quality assessment results showed high correlation with MOS.

Conclusions:

• The novel cGAN-based method provides an effective and reliable approach for MMIF quality assessment.
• This technique can significantly improve the trustworthiness of fused medical images in clinical practice.
• The findings pave the way for more robust evaluation metrics in medical image fusion.