Explainable fMRI-based brain decoding via spatial temporal-pyramid graph convolutional network.

Area of Science:

• Cognitive Neuroscience
• Neural Engineering
• Machine Learning

Background:

• Brain decoding using fMRI is crucial but faces challenges with low performance and poor explainability in current ML models.
• Existing methods struggle to capture complex spatial-temporal dynamics of neural activity effectively.

Purpose of the Study:

• To develop a biologically inspired architecture for improved fMRI-based brain decoding.
• To enhance the explainability of brain decoding models by identifying task-relevant brain regions.

Main Methods:

• Proposed Spatial Temporal-pyramid Graph Convolutional Network (STpGCN) to model spatial-temporal graph representations of brain activity.
• Incorporated multi-scale spatial-temporal and bottom-up pathways mimicking brain information processing.
• Introduced BrainNetX, a sensitivity analysis method for annotating task-related brain regions.

Main Results:

• STpGCN achieved significantly higher brain decoding performance compared to baseline models on fMRI data from 23 cognitive tasks (Human Connectome Project S1200).
• BrainNetX successfully identified task-relevant brain regions, aiding in the interpretation of decoding outcomes.
• Hierarchical structure of STpGCN demonstrated contributions to model explainability, robustness, and generalization.

Conclusions:

• The STpGCN architecture offers a promising approach for high-performance and explainable fMRI-based brain decoding.
• The findings provide insights into neural information representation across diverse cognitive tasks.
• This work paves the way for advancements in understanding brain function and developing brain-computer interfaces.