Benchmarking Multimodal Workload Classification: Effects of Modality, Validation Protocol, and Segmentation Contrast...

Physiology-based mental workload classification is hard to compare across studies because task design, preprocessing, segmentation, and validation protocols vary widely. Using OpenNeuro ds007262, an open multimodal arithmetic dataset of synchronised 19-channel 10-20 system electroencephalography (EEG), electrocardiography (ECG), and pupillometry data from 18 released participants (16 retained after participant-level quality control for downstream modelling) spanning seven objective difficulty bands plus baseline fixation, we present a reproducible end-to-end machine learning pipeline for graded workload classification. The pipeline standardises participant-level quality control, trial-aligned windowing, modality-specific preprocessing and feature extraction (153 EEG, 18 ECG, and 25 pupillometry features), and supervised evaluation under three validation protocols (within-participant, pooled-stratified, and group-holdout) over eleven models and five class scenarios. Fused representations generally performed best; EEG was the strongest unimodal modality, and classical models outperformed deep models in most feature-based conditions. The best 6 s pipeline reached a balanced accuracy of 0.635; under denser 3 s overlap segmentation, the best pipeline reached 0.718. Mean balanced accuracy across 60 matched cells rose from 0.324 to 0.380, with gains concentrated in within-participant and pooled-stratified evaluation rather than strict unseen-participant transfer. The pipeline provides a transparent benchmark framework for fine-grained physiological workload modelling.