Taking a Deeper Look at the Brain: Predicting Visual Perceptual and Working Memory Load From High-Density fNIRS Data

Area of Science:

• Neuroscience
• Cognitive Science
• Biomedical Engineering

Background:

• Predicting workload using physiological sensors is challenging due to small datasets and limited brain channel locations.
• Existing methods struggle with model transferability across participants, tasks, and sessions.
• High-density functional near-infrared spectroscopy (fNIRS) offers a promising avenue for workload prediction.

Purpose of the Study:

• To introduce a novel method for modeling large-scale, cross-participant, and cross-session fNIRS data for workload prediction.
• To leverage cognitive load theory and advanced deep learning techniques for improved workload classification.
• To develop a model capable of simultaneously predicting multiple levels of cognitive load.

Main Methods:

• Utilized high-density functional near-infrared spectroscopy (fNIRS) data from multiple participants and sessions.
• Employed a Convolutional Neural Network (CNN) combined with a Bi-Directional Gated Recurrent Unit (BiGRU) incorporating an attention mechanism.
• Implemented self-supervised label augmentation (SLA) and a multi-label classification scheme to predict simultaneous working memory load (WML) and visual processing load (VPL).
• Evaluated model performance using leave-one-participant-out cross-validation (LOOCV) and 10-fold cross-validation.

Main Results:

• Achieved high F1-scores for binary classification (WML: 0.9179, VPL: 0.8907) using LOOCV.
• Attained F1-scores of 0.7972 (WML) and 0.7968 (VPL) for multi-level classification via LOOCV.
• Demonstrated strong performance with 10-fold cross-validation for multi-level classification (WML: 0.7742, VPL: 0.7741).
• The CNN-BiGRU-SLA model effectively learned and classified different levels of WML and VPL across participants.

Conclusions:

• The proposed CNN-BiGRU-SLA model demonstrates robust performance in predicting cognitive workload levels from fNIRS data.
• The method shows significant potential for generalizability across participants and sessions, overcoming limitations of previous approaches.
• This work advances the field of workload prediction using neuroimaging techniques, with implications for human-computer interaction and cognitive monitoring.