3DFE-Net: Three-dimensional fusion enhancement network based on multi-attention mechanism for multi-modal magnetic...

Nowadays, the research of image fusion methods focuses on two-dimensional medical images, and almost no three-dimensional medical image fusion methods based on deep learning have been proposed. However, 3D image fusion is significant in clinical diagnosis. Therefore, this paper proposed a 3D medical image fusion enhancement network (3DFE-Net) for the gap in deep learning. 3DFE-Net included a feature extraction module, a multi-attention fusion module, and a feature reconstruction module. Firstly, multi-receptive field convolution blocks (MRFC) and multi-receptive field bottleneck blocks (MRFB) were devised instead of the traditional convolutional blocks to extract features of multiple receptive fields. Then, the multi-attention fusion module was designed using channel attention, self-attention, and spatial attention to make the network focus on the critical information in source images. Finally, the 3D fused image was obtained by the feature reconstruction module. In addition, a multivariate loss function was proposed for network training so that the fused image retains more edge structural information and texture details. MR-T1ce/MR-T2 fusion experiments show that, compared with the traditional method, 3DFE-Net improved the evaluation metrics EN (Information Entropy), MI (Mutual Information), SD (Standard Deviation), Qabf (Quality assessment of binary), and VIF (Visual Information Fidelity) by 0.0501, 0.1003, 5.2682, 0.1874, and 0.2129, respectively. 3DFE-Net can focus on the glioma lesion region in glioma slice fusion to achieve outstanding results and keep the structural information in MR-T1ce and the brightness information in MR-T2 well in normal slices. In qualitative and quantitative evaluations, 3DFE-Net performs better than conventional methods.