3D-FuseNet: An innovative multimodal fusion network for enhancing survival prediction

Predicting the survival time of brain tumor patients remains a critical challenge in the medical prognosis. Existing methods often struggle to integrate heterogeneous data types, such as imaging and clinical records. This paper proposes a novel approach, the Three-Dimensional Feature Fusion Network (3D-FuseNet), designed to address these limitations by integrating high-dimensional imaging features with clinical variables through a late fusion strategy. 3D-FuseNet leverages artificial intelligence (AI) models to automatically extract features from 3D of four MRI sequence images and fuses them with key clinical variables to fully exploit their complementary nature. The network was evaluated using 10 widely-used optimizers and 5 machine learning (ML) methods to assess its performance comprehensively. In the best-case scenario, 3D-FuseNet achieved a mean absolute error (MAE) that is 4% lower than the best baseline model. Furthermore, it reduced the MAE by 2.7% compared to the lowest MAE obtained using individual models such as ResNet50, Unet3D encoder, and Vision Transformer. By effectively integrating multimodal data sources, 3D-FuseNet has demonstrated competitive performance in predicting survival in brain tumor patients, achieving moderate yet consistent improvements over the evaluated baseline. The proposed method outperforms existing architectures and is publicly available at: https://github.com/AIPMLab/3D-FuseNet.