Capsule Attention for Multimodal EEG-EOG Representation Learning With Application to Driver Vigilance Estimation.

Area of Science:

• Neuroscience
• Transportation Safety Engineering
• Computer Science

Background:

• Driver vigilance is crucial for transportation safety.
• Real-time monitoring of driver vigilance can prevent accidents caused by distraction or impairment.
• Wearable brain-computer interface devices offer a promising avenue for continuous vigilance assessment.

Purpose of the Study:

• To propose a novel multimodal architecture for in-vehicle driver vigilance estimation.
• To leverage Electroencephalogram (EEG) and Electrooculogram (EOG) signals for enhanced vigilance monitoring.
• To introduce a capsule attention mechanism integrated with a Long Short-Term Memory (LSTM) network.

Main Methods:

• Developed a deep LSTM network to learn hierarchical dependencies in multimodal data.
• Implemented a capsule attention mechanism to focus on salient features in the learned representations.
• Investigated the impact of capsule attention parameters and different frequency bands/brain regions.

Main Results:

• The proposed multimodal architecture achieved state-of-the-art performance in driver vigilance estimation.
• The capsule attention mechanism significantly improved the model's discriminative ability.
• Experimental analysis confirmed the robustness and effectiveness of the approach, outperforming baseline methods.

Conclusions:

• The novel multimodal architecture with capsule attention offers a robust and effective solution for driver vigilance estimation.
• The findings highlight the potential of integrating EEG and EOG signals with advanced deep learning for improving road safety.
• Further analysis confirmed the advantages of multimodality and noise robustness in real-world driving scenarios.