A Hybrid Multimodal Emotion Recognition Framework for UX Evaluation Using Generalized Mixture Functions.

Area of Science:

• Affective computing
• Human-computer interaction (HCI)
• Artificial intelligence (AI)
• User experience (UX)

Background:

• Multimodal emotion recognition is crucial for affective computing and HCI applications.
• Current ensemble systems use static weights, limiting performance due to missing data and class variations.
• An effective weighting scheme is needed to improve discrimination between emotional modalities.

Purpose of the Study:

• To develop a hybrid multimodal emotion recognition (H-MMER) framework.
• To improve emotion recognition accuracy by assigning dynamic weights to different modalities using generalized mixture (GM) functions.
• To enhance the robustness of emotion classification systems for UX measurement.

Main Methods:

• Implemented a multi-view learning approach for unimodal emotion recognition.
• Introduced multimodal feature fusion and decision-level fusion using GM functions.
• Evaluated the framework on four emotional states: Happiness, Neutral, Sadness, and Anger.

Main Results:

• The proposed H-MMER framework achieved an average accuracy of 98.19% in modeling emotional states.
• GM functions effectively assigned dynamic weights, improving discrimination between modalities.
• Significant performance gains were observed compared to traditional emotion recognition approaches.

Conclusions:

• The H-MMER framework demonstrates high accuracy and robustness in identifying emotional states.
• Dynamic weighting using GM functions is effective for multimodal emotion recognition.
• The framework offers a promising solution for affective services in HCI and UX evaluation.