A multi-scale pyramid residual weight network for medical image fusion

Yiwei Liu ¹, Shaoze Zhang ¹, Yao Tang ²

⁰Center for Biomedical Imaging Research, School of Biomedical Engineering, Tsinghua University, Beijing, China.

Quantitative Imaging in Medicine and Surgery +

|

March 31, 2025

View abstract on PubMed

Summary

A new deep learning model, LYWNet, effectively fuses medical images, preserving both functional and structural details. This advanced multi-modal image fusion enhances diagnostic accuracy and surgical navigation by overcoming limitations of current methods.

Area Of Science

Medical Imaging
Deep Learning
Image Fusion

Background

Medical imaging faces challenges in simultaneously capturing functional and structural information.
Multi-modal image fusion integrates complementary data from different imaging types for enhanced diagnosis and navigation.
Existing deep learning fusion methods struggle to balance high-frequency details and low-frequency context, often degrading fine details.

Purpose Of The Study

To develop a novel deep learning method for accurate multi-modal medical image fusion.
To address the limitations of current methods in preserving both high-frequency and low-frequency information.
To improve the quality and precision of fused medical images for clinical applications.

Main Methods

A novel convolutional neural network (CNN), LYWNet, was proposed.
LYWNet employs a data preprocessing module, a feature extraction module with multi-scale pyramid residual weight blocks, and an image reconstruction module using feature distillation.
The architecture is designed to effectively integrate high-frequency details with low-frequency context.

Main Results

LYWNet successfully retained high-frequency details while integrating low-frequency contextual information.
The fused images showed reduced differences compared to original images.
Quantitative metrics demonstrated performance across various modalities: SPECT-MRI (SSIM: 0.5592, PSNR: 17.3594), PET-MRI (SSIM: 0.5195, PSNR: 14.5324), and MRI-CT (SSIM: 0.5376, PSNR: 13.9202).

Conclusions

LYWNet effectively integrates high-frequency and low-frequency information, surpassing existing deep learning fusion methods.
The method preserves crucial functional metabolic and anatomical texture details.
This technique offers a valuable tool for enhancing clinical diagnosis and surgical navigation.

Keywords:

Medical image fusion attention multiscale residual unsupervised learning