Bimodal EEG-fNIRS and Deep Learning for Classifying Intensity-Dependent Cortical Auditory Evoked Responses

Detection of intensity-dependent cortical auditory evoked responses using electroencephalography (EEG) is essential in clinical audiology and research on neurological disorders. While EEG remains the gold standard for monitoring brain activity, functional near-infrared spectroscopy (fNIRS) has emerged as a complementary modality, offering distinct insights into cortical responses. This study investigates the added value of fNIRS in classifying cortical responses to auditory stimuli across five intensity levels. Two classification models were developed: the TS-model, based on a convolutional neural network (CNN) with raw time-series (TS) inputs, and the F-model, based on a multi-layer perceptron (MLP) using features (F) extracted from the time-series. Both models were evaluated using bimodal EEG-fNIRS and unimodal EEG data, and compared against three conventional machine learning classifiers. Results showed that bimodal EEG-fNIRS inputs consistently outperformed unimodal EEG inputs across all models, highlighting the complementary information provided by fNIRS. The TS-model, which utilised raw time-series data, achieved the highest performance: 92.2% accuracy with bimodal input versus 79.3% with unimodal input. AUC and F1-score also improved with bimodal input, reaching 94.4% vs 80.5% and 89.6% vs 77.5%, respectively. These findings suggest that deep learning-based time-series analysis effectively captures intrinsic patterns critical for distinguishing cortical responses to varying auditory intensities. Leveraging multimodal neural data may enhance clinical assessments in hearing and neurological research.