Multimodal EEG and Eye-Tracking Assessment of Perceived Mental Workload and Situation Awareness in a Visual Display...

Visual display terminal (VDT) tasks are common in safety-critical systems, where operators must continuously monitor dynamic information and maintain situation awareness (SA). This study examined whether EEG and eye-tracking technologies could support the assessment of SA and perceived mental workload in VDT tasks. Short-term incentive conditions were included only as an exploratory contextual factor. Thirty participants completed a SATEST-based VDT simulation task while subjective scales, behavioral performance, EEG signals, and eye-tracking data were collected during the perception, comprehension, and projection stages of SA. Physiological indicators associated with SA differences and workload-related patterns were examined using SART-based and NASA-TLX-based grouping, supplemented by continuous-score Spearman correlation and exploratory linear regression analyses. The incentive analysis was exploratory, underpowered for small-to-medium effects, and was not designed to validate an incentive-based cognitive-state regulation strategy. Saccade-related eye-tracking indicators differed between relatively higher-SA and lower-SA groups across several stages (p = 0.0043 to 0.0457, Cohen's d = -0.670 to -1.002), while fronto-central relative theta power showed an SA-related difference during the SA1 perception stage (p = 0.0315, Cohen's d = -0.746). Relatively higher perceived workload was associated with lower SART scores (p = 0.0187), lower SA2 accuracy (p = 0.0175), larger SA3 angular error (p = 0.0151), and lower frontal theta-band and alpha-band absolute power across selected stages (p = 0.0094 to 0.0294). The exploratory incentive analysis showed only a limited stage-specific difference in SA1 distance error, with no reliable differences in overall SART score, SA2 accuracy, SA3 angular error, or NASA-TLX score. Thus, the findings provide insufficient evidence that the present short-term incentive manipulation produced reliable cognitive-state regulation. Overall, these findings provide preliminary support for multimodal physiological assessment in VDT tasks and suggest that the identified physiological measures should be regarded as candidate indicators requiring further validation.