Objective Assessment of Cognitive Workload in Surgery: A Systematic Review

To systematically review technologies that objectively measure cognitive workload (CWL) in surgery, assessing their psychometric and methodological characteristics.

Surgical tasks involving concurrent clinical decision-making and the safe application of technical and non-technical skills require a substantial cognitive demand and resource utilization. Cognitive overload leads to impaired clinical decision-making and performance decline. Assessing CWL could enable interventions to alleviate burden and improve patient safety.

Ovid MEDLINE, OVID Embase, the Cochrane Library, and IEEE Xplore databases were searched from inception to August 2023. Full-text, peer-reviewed original studies in a population of surgeons, anesthesiologists or interventional radiologists were considered, with no publication date constraints. Study population, task paradigm, stressor, cognitive load theory (CLT) domain, objective and subjective parameters, statistical analysis, and results were extracted. Studies were assessed for (1) definition of CWL; (2) details of the clinical task paradigm; and (3) objective CWL assessment tool. Assessment tools were evaluated using psychometric and methodological characteristics.

A total of 10,790 studies were identified; 9004 were screened; 269 full studies were assessed for eligibility, of which 67 met inclusion criteria. The most widely used assessment modalities were autonomic (32 eye studies and 24 cardiac). Intrinsic workload (eg, task complexity) and germane workload (effect of training or expertize) were the most prevalent designs investigated. CWL was not defined in 30 of 67 studies (44.8%). Sensitivity was greatest for neurophysiological instruments (100% EEG, 80% fNIRS); and across modalities accuracy increased with multisensor recordings. Specificity was limited to cardiac and ocular metrics, and was found to be suboptimal (50% and 66.67%). Cardiac sensors were the least intrusive, with 54.2% of studies conducted in naturalistic clinical environments (higher ecological validity).

Physiological metrics provide an accessible, objective assessment of CWL, but dependence on autonomic function negates selectivity and diagnosticity. Neurophysiological measures demonstrate favorable sensitivity, directly measuring brain activation as a correlate of cognitive state. Lacking an objective gold standard at present, we recommend the concurrent use of multimodal objective sensors and subjective tools for cross-validation. A theoretical and technical framework for objective assessment of CWL is required to overcome the heterogeneity of methodological reporting, data processing, and analysis.