U-Patch GAN: A Medical Image Fusion Method Based on GAN.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Multimodal medical imaging aids diagnosis but fusion algorithms often lose source image information.
• Accurate fusion of multimodal brain images is crucial for complex diagnoses.

Purpose of the Study:

• To develop an end-to-end generative adversarial network (GAN) model for high-quality, self-supervised multimodal brain image fusion.
• To enhance the retention of functional and structural information in fused medical images.

Main Methods:

• Developed U-Patch GAN, utilizing U-net as generator and PatchGAN discriminator for high-frequency information focus.
• Applied spectral norm for Lipschitz continuity and introduced novel adversarial and feature losses (F-norm based).
• Evaluated on public datasets, comparing single-slice and continuous-slice image fusion against six mainstream methods.

Main Results:

• The U-Patch GAN model significantly enhanced fused image quality compared to existing methods.
• Clinical utility assessments confirmed the practical value of the generated fused images.
• Verified the effectiveness of proposed adversarial and feature loss functions.

Conclusions:

• The U-Patch GAN model offers superior multimodal brain image fusion, improving diagnostic capabilities.
• The proposed loss functions and network architecture effectively preserve source image details.
• This approach advances self-supervised learning in medical image analysis.