Multimodal Autoencoder Predicts fNIRS Resting State From EEG Signals.

Area of Science:

• Neuroscience
• Biomedical Engineering
• Artificial Intelligence

Background:

• Epilepsy affects brain activity, necessitating advanced monitoring techniques.
• Multimodal brain imaging combining electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) offers comprehensive insights.
• Predicting hemodynamic responses from neural oscillations is a key challenge in brain-computer interfaces.

Purpose of the Study:

• To develop and evaluate a deep learning architecture for predicting fNIRS signals from EEG data.
• To investigate the predictive power of different EEG frequency bands on fNIRS signals in epileptic patients.
• To explore the potential of this multimodal approach for understanding brain dynamics in epilepsy.

Main Methods:

• A multimodal sequence-to-sequence autoencoder integrating Long Short-Term Memory (LSTM) units and Convolutional Neural Networks (CNNs).
• Training the model on simultaneous EEG and fNIRS recordings from 40 epileptic patients.
• Hierarchical feature extraction from EEG full spectra and specific frequency bands (e.g., gamma band).
• Validation using seed-based functional connectivity analysis.

Main Results:

• The deep learning model successfully predicted fNIRS signals from EEG data without prior assumptions.
• Higher frequency EEG ranges, especially the gamma band, were found to be highly predictive of fNIRS signals.
• Functional connectivity analysis confirmed similarities between experimental fNIRS data and model reconstructions.
• Demonstrated the predictive capability of EEG power spectrum amplitude modulation of frequency oscillations.

Conclusions:

• This study presents the first evidence of predicting brain hemodynamics (fNIRS) from neural data (EEG) in resting epileptic brains using deep learning.
• The findings highlight the significance of EEG gamma band activity in decoding hemodynamic responses.
• The developed architecture offers a novel approach for multimodal brain signal analysis in neurological disorders.