Driving-Related Cognitive Abilities Prediction Based on Transformer's Multimodal Fusion Framework

Area of Science:

• Cognitive Science
• Neuropsychology
• Traffic Safety Engineering

Background:

• Increasing urban road complexity and traffic volume necessitate advanced traffic safety measures.
• Existing research on driver assessment focuses on attention and reaction time, neglecting comprehensive cognitive evaluations.
• A gap exists in quantitatively assessing cognitive abilities crucial for driving in complex environments.

Purpose of the Study:

• To identify and quantitatively evaluate ten key cognitive components influencing driving decision-making, execution, and psychological states.
• To develop a novel computational model for predicting driving-related cognitive abilities using multimodal data.
• To provide a new methodology for driving risk assessment and traffic safety strategy development.

Main Methods:

• Analysis of driver actions from video footage in simulated driving scenarios.
• Collection of multimodal data, including physiological (ECG, EDA) and non-physiological (Eye Tracking) metrics.
• Development of a dual-branch Transformer network to extract temporal features from integrated multimodal data for cognitive ability prediction.

Main Results:

• The developed dual-branch Transformer network achieved high predictive accuracy for driving-related cognitive abilities.
• Experimental validation on a multimodal driving dataset yielded an Accuracy (ACC) of 0.9908 and an F1-score of 0.9832.
• The model effectively integrates behavioral and physiological data for cognitive assessment.

Conclusions:

• The proposed method offers a novel and effective approach to assessing cognitive abilities in drivers.
• This technique successfully combines behavioral analysis with secondary task performance for cognitive evaluation.
• The findings contribute significantly to advancing driving risk assessment and developing enhanced traffic safety strategies.