Concurrent Modeling of Naturalistic Functional Brain Networks: A Four-Dimensional Multi-Pattern Spatio-temporal...

Modeling the spatiotemporal patterns of whole-brain functional networks (FBNs) using functional magnetic resonance imaging (fMRI) is crucial for understanding brain function. Although existing methods, either shallow or deep models, have achieved promising outcomes, they lack the capability to concurrently extract multiple target FBNs while fully leveraging the inherent four-dimensional (4D) features of fMRI data. In this study, we propose a Multi-Pattern Spatiotemporal Hybrid Attention 4D CNN model (MSTHA-4DCNN) to concurrently capture the spatiotemporal patterns of multiple FBNs, building upon the rich spatial and temporal characteristics embedded in 4D fMRI data. The MSTHA-4DCNN extracts spatial patterns through the Multi-Pattern Spatial Attention 4D CNN (MSA-4DCNN), and subsequently incorporates Multi-Pattern Temporal Guided Attention Network (MT-GANet) to model temporal representations guided by the derived spatial patterns. We train the proposed model on a naturalistic fMRI dataset, and evaluate its generalizability on an independent public dataset from Cambridge Centre for Ageing and Neuroscience (Cam-CAN). The experimental results indicate that MSTHA-4DCNN exhibits promising performance and generalization ability in concurrently and effectively identifying spatiotemporal patterns of FBNs, outperforming other state-of-the-art models and offering a potent tool for advancing our understanding of complex neural processes.