Theories and Methods for Labeling Cognitive Workload: Classification and Transfer Learning.

Area of Science:

• Cognitive science
• Machine learning
• Human-computer interaction

Background:

• Developing classifiers for operator functional states requires addressing data-centric questions regarding learning approaches and data preparation.
• Supervised learning for cognitive load data necessitates effective labeling strategies.

Purpose of the Study:

• To explore and compare three distinct methods for labeling cognitive states for three-state classification.
• To evaluate the impact of different labeling techniques on classifier performance and transferability across individuals and tasks.

Main Methods:

• Investigated three labeling methods: tertiary split of trial difficulty, mixed-effects stress-strain curve for performance asymptotes, and mixed-effects Rasch model for item response theory capacity limits.
• Utilized elastic net and random forest classifiers to assess labeling approach strength via Area Under the Curve (AUC) from Receiver Operating Curves (ROCs).
• Tested techniques across two participant groups and two tasks to evaluate cross-person and cross-task transfer.

Main Results:

• The Rasch model labeling method, when paired with a random forest classifier, yielded the best model fits.
• This combination demonstrated significant evidence of both cross-person and cross-task transferability.

Conclusions:

• The mixed-effects Rasch model provides a robust method for labeling cognitive states for supervised learning.
• Random forest classifiers, combined with Rasch model-derived labels, offer superior performance and generalizability for operator functional state classification.