Decoding Mental Effort in a Quasi-Realistic Scenario: A Feasibility Study on Multimodal Data Fusion and...

Area of Science:

• Cognitive Science
• Human-Computer Interaction
• Biomedical Engineering

Background:

• Human performance is influenced by available mental resources, which fluctuate with task demands and environmental factors.
• Monitoring cognitive load in naturalistic settings requires integrating task-induced demands with situational influences.
• Previous methods for assessing mental effort often lack generalizability across individuals and real-world scenarios.

Purpose of the Study:

• To investigate the feasibility of decoding experienced mental effort using a multimodal approach.
• To develop and test a machine learning architecture for combining physiological signals to predict mental effort.
• To establish a foundation for generalized, cross-individual mental state monitoring in realistic applications.

Main Methods:

• A multimodal study involving 18 participants performing a demanding task with emotional distraction.
• Simultaneous recording of respiratory, ocular, cardiac, and brain activity (functional near-infrared spectroscopy - fNIRS).
• Development of a multimodal machine learning architecture including feature engineering, optimization, and cross-subject classification.

Main Results:

• The multimodal machine learning architecture successfully decoded experienced mental effort.
• The approach reliably distinguished between two distinct levels of mental effort.
• The proposed method demonstrated reduced overfitting and enhanced classification accuracy.

Conclusions:

• Multimodal physiological data combined with machine learning can effectively predict mental effort.
• This approach offers a promising pathway for developing generalized mental state monitoring systems.
• The findings support the potential for real-time cognitive state assessment in various applications.