CT-Net: an interpretable CNN-Transformer fusion network for fNIRS classification.

Area of Science:

• Neuroscience
• Biomedical Engineering
• Machine Learning

Background:

• Functional near-infrared spectroscopy (fNIRS) is a key optical neuroimaging technique for brain activity recognition.
• Deep learning models are increasingly applied to fNIRS data classification challenges.

Purpose of the Study:

• To introduce CT-Net, a novel deep learning model combining convolutional neural networks and Transformers for fNIRS-based mental arithmetic task classification.
• To enhance data utilization and feature learning by exploring novel data representations, specifically a temporal-level combination of raw chromophore signals.

Main Methods:

• Developed CT-Net, integrating convolutional neural network and Transformer architectures.
• Implemented a temporal-level combination of two raw chromophore signals for enriched feature learning.
• Evaluated model performance on two open-access fNIRS datasets.

Main Results:

• Achieved high classification accuracies of 98.05% and 77.61% on two distinct datasets.
• Demonstrated model interpretability using gradient-weighted class activation mapping, showing consistency with brain activity patterns during mental arithmetic tasks.

Conclusions:

• CT-Net shows significant feasibility and interpretability for decoding mental arithmetic tasks using fNIRS data.
• The proposed data representation strategy effectively improves feature learning and model performance.