SMFF-Net: Spatiotemporal-frequency Multi-domain Feature Fusion Network for EEG-based brain state detection

Area of Science:

• Neuroscience
• Machine Learning
• Traffic Safety

Background:

• Driver fatigue poses significant risks to road safety.
• Current electroencephalography (EEG)-based deep learning methods for fatigue detection lack cross-domain feature interaction and neurophysiological interpretability.

Purpose of the Study:

• To develop a novel deep learning model for accurate and interpretable driver fatigue detection using EEG.
• To address limitations in existing EEG-based fatigue detection by integrating spatiotemporal and frequency domain features.

Main Methods:

• Proposed the Spatiotemporal-frequency Multi-domain Feature Fusion Network (SMFF-Net) utilizing Convolutional Neural Networks (CNNs) and Graph Neural Networks (GCNs).
• Incorporated an Enhanced Cognitive Importance Modulation (ECIM) module for optimizing domain-specific features.
• Implemented an Entropy-Guided Gated Fusion (EGGF) module for adaptive feature integration based on uncertainty and complementarity.

Main Results:

• SMFF-Net achieved high performance on SEED-VIG (98.54% accuracy, 98.39% F1-score) and SADT (99.74% accuracy, 99.73% F1-score) datasets.
• Model analysis revealed associations between discriminative features and fatigue-related EEG alterations: temporal dynamics destabilization, brain connectivity reconfiguration, and spectral rhythm slowing.
• Demonstrated superior performance compared to state-of-the-art methods.

Conclusions:

• Coordinated multi-domain representation modeling in SMFF-Net significantly enhances EEG-based driver fatigue detection effectiveness.
• The model offers neurophysiologically interpretable characterizations of brain state alterations associated with fatigue.
• SMFF-Net represents a promising advancement for intelligent transportation systems and driver safety.