A Novel Vision Transformer Based Multimodal Fusion Approach for Clinical MDD Diagnosis Using EEG and Audio Signals.

Area of Science:

• Neuroscience
• Artificial Intelligence
• Signal Processing

Background:

• Major Depressive Disorder (MDD) diagnosis is challenging due to subjective clinical assessments.
• There is a critical need for objective, automated tools for reliable MDD detection.
• Current diagnostic methods lack the precision required for timely intervention.

Purpose of the Study:

• To develop a novel multimodal framework for accurate Major Depressive Disorder (MDD) detection.
• To integrate electroencephalography (EEG) and audio signals for enhanced diagnostic capabilities.
• To create an automated system for objective and efficient MDD assessment.

Main Methods:

• EEG signals were processed and converted into 2D time-frequency (T-F) representations using Superlet Transform.
• Audio signals were transformed into Mel-spectrograms.
• A novel Vision Transformer (ViT) architecture processed modality-specific data, followed by feature fusion and classification.

Main Results:

• The multimodal framework achieved 98.86% accuracy, 98.32% F1-score, and 0.9403 MCC on the MODMA dataset.
• The proposed method significantly outperformed unimodal diagnostic approaches.
• The system demonstrated superior performance in detecting Major Depressive Disorder.

Conclusions:

• The developed multimodal framework offers a highly accurate and objective method for MDD detection.
• The architecture is suitable for deployment in Internet of Medical Things (IoMT) systems for remote, low-latency diagnosis.
• This approach enhances the accessibility and real-time decision-making capabilities in mental healthcare.