A multimodal deep learning framework for functional brain network classification in rs-fMRI

To automate the classification of functional brain networks in epilepsy patients using resting-state functional magnetic resonance imaging (rs-fMRI). The study introduces a deep learning framework that leverages spatial and temporal features to classify Independent Component Analysis (ICA)-derived networks into 11 functionally distinct classes, including seizure onset zone (SoZ), resting-state networks (RSNs), and artifact/noise. A hybrid deep learning architecture was developed combining a 3D Convolutional Neural Network (3D-CNN) to extract spatial features (SF) and a Long Short-Term Memory (LSTM) network to capture temporal dynamics from time-domain (TS) and frequency-domain (FS) signals. These multi-domain features were concatenated and classified into 11 distinct ICA component types. An ablation study assessed the individual and combined contributions of spatial, temporal, and spectral features. Additionally, expert neurologists independently rated four representative cases to qualitatively validate the model's interpretability and clinical relevance. The baseline 3D CNN (SF) model achieved an overall accuracy of 69% with a sensitivity of 0.52 and a ROC AUC of 0.76. Incorporating frequency-domain signals (SF + FS) enhanced sensitivity to 0.54 and improved the ROC AUC to 0.78 while maintaining a similar accuracy. Combining both time-domain and frequency-domain signals (SF + TS + FS) yielded the highest accuracy at 70%. At the class level, the Noise class consistently demonstrated robust performance (up to 0.94), whereas the temporal lobe network class Temporal class exhibited lower scores (0.14-0.24) across all configurations. Our results demonstrate that this data-driven framework can effectively automate the classification of rs-fMRI-derived functional brain networks including SoZ thereby reducing subjectivity and workload in clinical review. The inclusion of spatial, temporal, and spectral information enables a richer and more nuanced classification that supports downstream applications in epilepsy surgical planning.