Multi-modal depression detection based on emotional audio and evaluation text.

Area of Science:

• Psychiatry
• Computational Linguistics
• Signal Processing

Background:

• Early depression detection is crucial for effective patient treatment.
• Current screening methods for depression are often inefficient.
• Developing advanced depression identification technology holds significant clinical value.

Purpose of the Study:

• To propose and validate a novel experimental method for depression detection using audio and text data.
• To investigate the efficacy of a new text reading experiment, the Segmental Emotional Speech Experiment (SESE), in eliciting emotional changes.
• To develop and evaluate an efficient multi-modal deep learning model for depression recognition.

Main Methods:

• A new experimental paradigm, the Segmental Emotional Speech Experiment (SESE), was designed to rapidly induce emotional variations in subjects.
• Low-level audio features (384-dimensional) were extracted during SESE to analyze emotional differences.
• A multi-modal fusion approach combined DeepSpectrum features and word vector features, utilizing deep learning for depression detection.

Main Results:

• The SESE method demonstrated an improvement in depression recognition accuracy.
• Significant differences in low-level audio features were identified correlating with emotional states.
• The proposed multi-modal fusion model achieved a high accuracy of 0.912 and an F1 score of 0.906.

Conclusions:

• The Segmental Emotional Speech Experiment (SESE) offers a novel approach for researchers studying emotional dynamics and depression.
• A new, efficient multi-modal model for depression recognition has been developed, showing promising results.
• The findings highlight the potential of integrating audio and text analysis for improved mental health diagnostics.