"Are You Okay, Honey?": Recognizing Emotions Among Couples Managing Diabetes in Daily Life Using Multimodal...

Area of Science:

• Health Informatics
• Affective Computing
• Human-Computer Interaction

Background:

• Chronic disease management significantly impacts couples' emotional well-being.
• Current emotion assessment methods for couples are manual, time-consuming, and expensive.
• Existing emotion recognition research lacks data from couples' daily interactions.

Purpose of the Study:

• To develop and evaluate automated emotion recognition models for partners in couples managing chronic diseases.
• To investigate the feasibility of using real-world multimodal sensor data for emotion recognition in daily life.
• To improve understanding of emotional well-being within couples managing chronic conditions.

Main Methods:

• Collected 85 hours of multimodal smartwatch sensor data (speech, heart rate, accelerometer, gyroscope) and self-reported emotions from 26 partners (13 couples) with type 2 diabetes.
• Extracted physiological, movement, acoustic, and linguistic features from the collected data.
• Trained and evaluated machine learning models (Support Vector Machine, Random Forest) for recognizing self-reported valence and arousal.

Main Results:

• The best models achieved balanced accuracies of 63.8% for arousal and 78.1% for valence.
• Performance surpassed chance, prior work with similar populations, and partners' own perceptions.
• Demonstrated the effectiveness of using real-world sensor data for emotion recognition in couples.

Conclusions:

• Automated emotion recognition systems are feasible for couples managing chronic diseases using daily life data.
• These systems can provide insights into partners' emotional states, aiding well-being.
• Future interventions can leverage these technologies to support couples' emotional health during chronic illness management.