DAGM-fusion: A dual-path CT-MRI image fusion model based multi-axial gated MLP

Medical imaging technology provides a good understanding of human tissue structure. MRI provides high-resolution soft tissue information, and CT provides high-quality bone density information. By creating CT-MRI fusion images of complex diagnostic situations, experts can develop diagnoses and treatment plans more quickly and precisely. We propose a dual-path CT-MRI image fusion model based on multi-axial gated MLP to create high-quality CT-MRI fusion images. The model employs the feature fusion module SFT-block to effectively integrate detailed Local-Path information guided by global Global-Path information. The fusion is completed through triple constraints, namely global constraints, local constraints, and overall constraints. We design a multi-axial gated MLP module (Ag-MLP). The multi-axial structure maintains the computational complexity linear and increases MLP's inductive bias, allowing MLP to work in shallower or pixel-level small dataset tasks. Ag-MLP and CNN are combined in the network so that the model has both globality and locality. In addition, we design a loss calculation method based on image patches that adaptively generates weights for each patch based on image pixel intensity. The details of the image are efficiently increased when patch-loss is used. Numerous studies demonstrate that the results of our model are superior to those of the latest mainstream fusion model, which are more in accordance with actual clinical diagnostic standards. The ablation studies successfully validate the performance of the model's constituent parts. It is worth mentioning that the model can also be excellently generalized to other modal image fusion tasks.