Predicting affective engagement and mental strain from prosodic speech features.

Area of Science:

• Psychology
• Linguistics
• Computer Science

Background:

• Emotional resilience and cognitive load are key to mental health.
• Traditional assessment methods are intrusive and lack real-time insights.
• Speech prosody offers a non-intrusive alternative for psychological assessment.

Purpose of the Study:

• To develop proxy measures for emotional resilience and cognitive load using speech prosody.
• To correlate speech patterns with self-reported engagement, enjoyment, boredom, and cognitive effort.
• To build machine learning models for predicting psychological states from speech.

Main Methods:

• Utilized the SEWA database for dyadic conversation recordings.
• Extracted and normalized prosodic features (pitch, intensity, voice activity).
• Employed regression and classification models correlating speech features with self-reports.

Main Results:

• Established a significant link between speech prosody and psychological states.
• Positive Affective Engagement (emotional resilience) and Perceived Mental Strain (cognitive load) are predictable via prosody.
• Language-specific variations impact emotional resilience prediction, while cognitive load prediction is consistent across English and German.

Conclusions:

• Speech prosody provides a novel, non-intrusive method for assessing psychological states.
• Tailored, multilingual models are crucial for accurate estimation of emotional resilience and cognitive load.
• Findings have implications for mental health monitoring, workload analysis, and human-computer interaction.