Bimodal EEG-fNIRS and Deep Learning for Classifying Intensity-Dependent Cortical Auditory Evoked Responses

Area of Science:

• Neuroscience
• Biomedical Engineering
• Audiology

Background:

• Electroencephalography (EEG) is crucial for detecting intensity-dependent cortical auditory evoked responses.
• Functional near-infrared spectroscopy (fNIRS) offers complementary insights into brain activity.
• Classifying auditory responses is vital for clinical audiology and neurological research.

Purpose of the Study:

• To investigate the added value of fNIRS combined with EEG for classifying cortical responses to auditory stimuli at different intensities.
• To compare deep learning models (CNN, MLP) using time-series and feature-based inputs with conventional classifiers.
• To evaluate the performance of unimodal EEG versus bimodal EEG-fNIRS data.

Main Methods:

• Developed two classification models: TS-model (CNN, raw time-series) and F-model (MLP, extracted features).
• Evaluated models using both unimodal EEG and bimodal EEG-fNIRS data.
• Compared performance against three conventional machine learning classifiers.

Main Results:

• Bimodal EEG-fNIRS inputs consistently outperformed unimodal EEG across all models.
• The TS-model achieved the highest accuracy (92.2% bimodal vs. 79.3% unimodal).
• Area Under the Curve (AUC) and F1-score also significantly improved with bimodal input.

Conclusions:

• Multimodal neural data (EEG-fNIRS) provides complementary information, enhancing the classification of auditory cortical responses.
• Deep learning-based time-series analysis effectively captures patterns for distinguishing responses to varying sound intensities.
• Integrating multimodal neural data can improve clinical assessments in hearing and neurological research.