Multimodal Fusion of EEG and Audio Spectrogram for Major Depressive Disorder Recognition Using Modified DenseNet121.

Area of Science:

• Neuroscience
• Machine Learning
• Signal Processing

Background:

• Major Depressive Disorder (MDD) diagnosis can be challenging.
• Electroencephalography (EEG) and audio signals offer potential biomarkers for mental health assessment.

Purpose of the Study:

• To develop a multimodal classification model integrating EEG and audio data for MDD identification.
• To evaluate the model's performance in detecting depressive tendencies.

Main Methods:

• Utilized the Multimodal open dataset for Mental Disorder Analysis (MODMA).
• Employed a pre-trained Densenet121 model with transfer learning.
• Extracted and concatenated features from EEG (Short-Time Fourier-Transform spectrograms) and audio (Mel-spectrograms) modalities.

Main Results:

• Achieved high classification performance: 97.53% Accuracy, 98.20% Precision, 97.76% F1 Score, and 97.32% Recall.
• The multimodal approach outperformed existing single-modality methods.
• A confusion matrix analysis confirmed the model's effectiveness.

Conclusions:

• The proposed multimodal classification approach is robust and effective for MDD assessment.
• This method shows significant potential for application in clinical diagnostics.
• Integration of EEG and audio data enhances depression classification accuracy.