Multimodal deep feature fusion with transformer for brain tumor classification from magnetic resonance imaging

Brain tumors (BTs) arise due to abnormal cell growth, which has a high mortality rate globally. Millions of lives can be saved through the timely identification of BT. Precise identification and segmentation of BTs are essential to enhance the precision of analysis and the efficiency of therapeutic strategies. Magnetic resonance imaging (MRI) is a broadly utilized analytical tool. Furthermore, deep learning (DL) has recently shown efficiency in addressing several computer vision tasks. Several DL-driven methods are implemented for BT segmentation and attained impressive outcomes. This study presents a Multimodal Deep Feature Fusion Framework for Automated Brain Tumor Detection and Segmentation (MDFF-ABTDS) model. This objective is to develop a multimodal DL that integrates feature fusion and transformer networks for the precise detection and segmentation of BTs from medical images. Initially, image pre-processing is performed using Contrast Limited Adaptive Histogram Equalization (CLAHE) and image normalization. Feature extraction is carried out through fusion models such as CapsNet, ResNet-50, and AlexNet. These extracted features are then passed to a bi-directional convolutional long short-term memory combined with transformer (TBConvL-Net) models to classify tumors and non-tumors effectively. Finally, the tumor is classified to identify its location using the nnUNet model for a precise segmentation process. A series of experimental analyses of the MDFF-ABTDS method portrayed a superior accuracy value of 98.91% over existing models under the BT MRI dataset.