Miniformer: A Minimalist Transformer for Brain Functional Networks Analysis.

Area of Science:

• Neuroscience
• Machine Learning
• Medical Diagnostics

Background:

• Estimating and classifying brain functional networks (BFNs) is crucial for early prediction of neurological and mental disorders.
• Traditional methods separate BFN estimation and classification, limiting joint optimization.
• Transformer models offer end-to-end learning for BFNs but suffer from large parameters and poor interpretability.

Purpose of the Study:

• To propose a minimalist Transformer architecture (Miniformer) for improved BFN estimation and classification.
• To address the challenges of large data requirements and the need for model interpretability in medical applications.
• To develop variants of Miniformer incorporating domain knowledge for enhanced fMRI signal analysis.

Main Methods:

• Introduced Miniformer by simplifying Transformer's self-attention projection matrices to a single diagonal matrix.
• Developed Miniformer variants with sparsity and smoothness constraints for fMRI signal processing.
• Evaluated Miniformer and its variants on three public datasets for brain disease diagnosis.

Main Results:

• Miniformer significantly reduces model parameters, mitigating overfitting and enhancing interpretability.
• The proposed variants effectively integrate domain knowledge (sparsity and smoothness) into BFN analysis.
• Experiments demonstrated that Miniformer and its variants achieve superior classification performance compared to existing methods.

Conclusions:

• Miniformer provides a computationally efficient and interpretable approach for BFN analysis.
• The model's design facilitates the integration of prior knowledge, crucial for medical AI.
• Miniformer and its variants show significant promise for early detection and diagnosis of brain disorders.