Spectral and Temporal Feature Learning With Two-Stream Neural Networks for Mental Workload Assessment

Area of Science:

• Neuroscience
• Cognitive Science
• Machine Learning

Background:

• Mental workload significantly impacts work efficiency and health, necessitating accurate assessment to prevent accidents.
• Electroencephalogram (EEG) spectral and temporal features are valuable for mental workload assessment.
• Fusing these EEG features offers richer distinguishing information for improved assessment.

Purpose of the Study:

• To propose a novel deep learning model, the two-stream neural network (TSNN), for fusing spectral and temporal EEG features.
• To automatically learn and fuse EEG features without prior knowledge, outperforming traditional methods.
• To enhance the accuracy and reliability of mental workload assessment.

Main Methods:

• Development of a two-stream neural network (TSNN) with separate spectral and temporal streams.
• Each stream utilizes convolutional neural networks (CNNs) and temporal convolutional networks (TCNs) to process EEG topographic maps.
• Fusion of spectral and temporal features is achieved by concatenating stream outputs before a fully connected layer.

Main Results:

• The TSNN achieved an average accuracy of 91.9% in a three-class mental workload classification task.
• Demonstrated significant improvement over baseline classifiers using hand-crafted features.
• Outperformed existing state-of-the-art deep learning methods for EEG classification.

Conclusions:

• The proposed TSNN is a promising structure for fusing spectral and temporal EEG features for mental workload assessment.
• The method offers a high-precision approach for applications during cognitive activities.
• This deep learning approach enhances the understanding and measurement of cognitive load.