Supporting the driver's perception cognition action loop using an embodied intelligent cockpit framework for explainable fatigue intervention

This study evaluated a driver-support framework combining multimodal fatigue detection, feature attribution, and state-specific auditory feedback during conditional automated driving.

Forty licensed drivers completed three counterbalanced 20-min simulator conditions: nonverbal warnings, generic speech, and explanatory speech. A fixed detector used electroencephalography (EEG), eye tracking, and vehicle dynamics. Shapley additive explanations (SHAP) selected a dominant cue, which a large language model translated into a spoken message. Fatigue labels were based on the Karolinska Sleepiness Scale and a 60-s vigilance task. Braking reaction time, standard deviation of lane position (SDLP), and time-to-collision (TTC) exposure were assessed during manual driving following takeover.

Compared with generic speech, explanatory speech was associated with a 0.192-s shorter mean braking reaction time (95% confidence interval: -0.223 to -0.161 s; Holm-adjusted p < 0.001) and a 0.085-m lower mean SDLP (95% confidence interval: -0.108 to -0.062 m; Holm-adjusted p < 0.001). The mean proportion of time with TTC < 1.5 s was 0.07 versus 0.10. Retrospective leave-one-subject-out evaluation yielded an area under the receiver-operating-characteristic curve of 0.85.

The explanatory warning package was associated with faster braking and more stable lane keeping than generic speech in this simulator study. Further studies should separate message content from acoustic properties and assess performance in independent driving settings.