Explainable spatio-temporal transformers for schizophrenia classification and activity analysis from functional MRI

Schizophrenia is a complicated neuropsychiatric disorder caused by disturbed brain connectivity and abnormal neural activation patterns. To identify these underlying neural mechanisms, computational models need to be reliable and interpretable to support objective diagnosis. Thus, the objective of this work is to develop an interpretable spatiotemporal deep learning model for classifying schizophrenia using functional magnetic resonance imaging (fMRI) data.

We introduce a proposed spatiotemporal transformer architecture that combines a 3D vision transformer (ViT) to capture spatial features with a temporal transformer that models dynamic brain activity over the course of fMRI time points. Each fMRI volume is processed individually to acquire a spatial representation, after which temporal dependencies are learned over the entire time series. Performance testing was conducted with schizophrenia and healthy control subjects. Model interpretability was performed by means of Grad-CAM and attention rollout analysis. Finally, group-level activation patterns were analyzed for neurobiological relevance.

The proposed model achieved a mean testing accuracy of 95.10 ± 3.76% (95% CI: [0.9043, 0.9977]) and a mean area under the curve (AUC) of 0.998 ± 0.004 across five stratified folds. Attention-based interpretability analyses have underlined marked differences in spatial and temporal patterns of activation between groups. Healthy controls showed focused and symmetric activation within the Default Mode Network (DMN), including posterior cingulate cortex (PCC) and precuneus, while schizophrenia subjects had more diffuse and posteriorly shifted activations extending into the parieto-occipital cortex, indicative of disrupted DMN connectivity and lower attentional coherence.

The proposed spatiotemporal transformer model would effectively capture both spatial and temporal characteristics of fMRI while retaining biologically meaningful interpretability. Group-level differences in attention patterns indicate disrupted DMN organization in schizophrenia. This model can be a strong candidate for a sturdy and interpretable computational tool in neuropsychiatric disorder analysis and computer-aided diagnosis.