High-Density Electroencephalography and Speech Signal Based Deep Framework for Clinical Depression Diagnosis

Area of Science:

• Neuroscience
• Computational Psychiatry
• Machine Learning

Background:

• Depression diagnosis often relies on subjective assessments, with existing objective methods focusing on moderate to severe cases.
• Early detection of mild depression is crucial for timely intervention and improved patient outcomes.
• Integrating multimodal data offers potential for enhanced diagnostic accuracy.

Purpose of the Study:

• To develop and validate a novel diagnostic approach for depression, particularly at its mild stage.
• To improve the accuracy of depression diagnosis by combining audio spectrogram and electroencephalography (EEG) signals.
• To create a publicly accessible diagnostic framework and source code.

Main Methods:

• Fusion of speech features (audio spectrogram) and multi-frequency EEG signals.
• Application of vision transformers and pre-trained networks on fused speech and EEG spectrum data.
• Extensive experimentation and validation using the Multimodal Open Dataset for Mental-disorder Analysis (MODMA).

Main Results:

• Achieved significant improvements in depression diagnosis performance for mild-stage patients.
• Reported high precision (0.972), recall (0.973), and F1 score (0.973) on the MODMA dataset.
• Demonstrated the efficacy of combining audio and EEG data for enhanced diagnostic capabilities.

Conclusions:

• The proposed multimodal approach significantly enhances the early diagnosis of mild depression.
• Combining speech and EEG signals with advanced machine learning models offers a promising avenue for objective mental health assessment.
• The developed web-based framework and open-source code facilitate broader research and clinical application.