Scepter: Weakly Supervised Framework for Spatiotemporal Dense Prediction of 4D Dynamic Brain Networks

Spatiotemporal brain dynamism is a complex phenomenon, characterized by dynamic patterns of neural activity that unfold across both space and time. However, capturing these dynamic patterns poses a formidable challenge due to the sheer complexity of neural interactions and the demand for advanced computational models. In this context, we have harnessed advances in computer vision and formulated this challenging issue as the weakly supervised spatiotemporal dense prediction of dynamic brain networks. To accomplish this, we have developed a novel framework for encoding spatiotemporal characteristics of functional magnetic resonance imaging (fMRI) data to densely predict dynamic brain networks, each encompassing 4D maps that vary over time and between subjects. The backbone of our framework is an isotropic model architecture that contains a deep stack of pre-activated ConvMixer modules. Furthermore, we introduce a strategy for generating prior information, which serves as weak supervision for training the model, since no benchmark currently exists for addressing the dynamic brain network issue and annotating fMRI data proves to be an expensive and inaccurate process. We also address some of the significant drawbacks in popular brain parcellation methods. Finally, our experimental results indicate the method's ability to generate plausible brain network maps that are highly dynamic and consistent with previous findings in brain dynamics. The proposed advancement in generating brain dynamic maps transcends the boundaries of conventional neuroscience research, ushering in a paradigm shift which facilitates the discovery of new perspectives on the complexity of brain function.