MRI-informed multi-modal deep learning for EEG source imaging with subject-specific head structure recognition

Electroencephalography (EEG) source imaging aims to reconstruct the origin of neural activity within the brain from non-invasive scalp recordings. However, it remains fundamentally challenging due to the inherent ill-posed nature of the inverse problem and substantial inter-subject anatomical variability. Traditional numerical methods and recent deep learning approaches often rely on strong priors or fixed head models, limiting their ability to generalize across diverse subjects. To address these limitations, we propose a subject-specific multi-modal deep learning framework that explicitly integrates temporal EEG signals with structural magnetic resonance (MR)-derived anatomical information, enabling robust EEG source localization across anatomically diverse unseen subjects.

Subject-specific head models derived from magnetic resonance (MR) images are incorporated into a deep neural network trained on simulated EEG datasets generated through realistic forward modeling. By jointly learning structural information from MRI and temporal dynamics from EEG, the proposed framework learns anatomy-aware representations that enable robust localization across previously unseen head anatomies.

In simulation, the proposed framework achieved a localization error of 5 mm while requiring only 1.64 ms for inference, consistently outperforming conventional inverse methods, representative deep learning-based EEG source imaging approaches, and the uni-modal EEG-only baseline. Furthermore, the proposed framework demonstrated robust localization performance across anatomically diverse unseen subjects, and its advantage over the uni-modal baseline was preserved on recorded intracranial-stimulation data, highlighting the benefit of incorporating subject-specific anatomical information.

These findings demonstrate that incorporating subject-specific anatomical information substantially improves the robustness and generalizability of EEG source imaging while enabling real-time inference, highlighting the potential of the proposed framework for personalized neuroimaging applications.