Emotion recognition in dance therapy driven by DanceEmoNet: a deep learning model based on facial expression and pose...

Area of Science:

• Computer Science
• Psychology
• Biomedical Engineering

Background:

• Accurate, real-time emotion recognition is crucial for dance therapy's mental health applications.
• Unimodal emotion recognition methods struggle with the complexity and temporal dynamics of emotional changes.
• Existing methods lack the precision needed for dynamic, real-world therapeutic settings.

Purpose of the Study:

• To develop an advanced multimodal emotion recognition model, DanceEmoNet, for dance therapy.
• To integrate facial expression and pose estimation for enhanced emotional state identification.
• To improve the precision and temporal accuracy of emotion recognition in dance contexts.

Main Methods:

• Proposed DanceEmoNet model integrating facial expression and pose estimation.
• Utilized YOLOv11 for object detection (face, pose), TriBAN and CNN-LSTM for feature extraction/temporal modeling.
• Employed Graph Convolutional Network (GCNC) module for multimodal feature fusion.

Main Results:

• DanceEmoNet demonstrated superior performance across multiple metrics compared to benchmark models.
• Achieved faster inference speeds (reduced latency) and lower computational costs (fewer FLOPs).
• Reported overall performance gains ranging from 5% to 10% in emotion recognition accuracy.

Conclusions:

• DanceEmoNet effectively captures complex emotional changes and dynamic dance movements.
• The model's multimodal approach offers significant advantages over unimodal methods.
• DanceEmoNet shows practical applicability for real-world deployment in mental health interventions.