Diagnosis of Schizophrenia Using Multimodal Data and Classification Using the EEGNet Framework

Area of Science:

• Neuroscience and Artificial Intelligence
• Clinical Psychology and Psychiatry

Background:

• Rising global stress and emotional difficulties contribute to severe mental disorders like schizophrenia.
• Schizophrenia presents complex positive, negative, and cognitive symptoms, complicating diagnosis and treatment.
• Accurate and early diagnosis of schizophrenia is crucial for effective intervention.

Purpose of the Study:

• To develop a system for identifying schizophrenia symptom severity using multimodal data.
• To classify schizophrenia patients using the advanced EEGNet framework.
• To integrate electrophysiological data with behavioral and affective cues for enhanced diagnostic accuracy.

Main Methods:

• Collected multimodal data including facial expressions, speech signals, and 14-channel electroencephalogram (EEG) recordings.
• Utilized validated assessment tools: Positive and Negative Symptoms Scale (PANSS), Brief Negative Symptom Scale (BNSS), and others.
• Employed photo elicitation and virtual reality (VR) stimuli to capture patient responses.
• Developed a novel method fusing normalized multimodal features into the EEGNet architecture.

Main Results:

• The EEGNet model demonstrated exceptional performance with 0.99 accuracy, 0.99 precision, 0.98 recall, and 0.99 F1-score for healthy individuals.
• For schizophrenia patients, the model achieved 0.98 precision, 0.99 recall, and 0.99 F1-score.
• The system achieved a Receiver Operating Characteristic Area Under the Curve (ROC AUC) of approximately 0.9989.

Conclusions:

• The developed multimodal EEGNet system shows significant promise for accurate schizophrenia diagnosis.
• The integration of EEG data with other modalities enhances classification performance.
• This approach offers a potential advancement in diagnostic tools for schizophrenia.