A Multimodal AI System for Out-of-Distribution Generalization of Seizure Identification

Artificial intelligence (AI) and health sensory data-fusion hold the potential to automate many laborious and time-consuming processes in hospitals or ambulatory settings, e.g. home monitoring and telehealth. One such unmet challenge is rapid and accurate epileptic seizure annotation. An accurate and automatic approach can provide an alternative way to label seizures in epilepsy or deliver a substitute for inaccurate patient self-reports. Multimodal sensory fusion is believed to provide an avenue to improve the performance of AI systems in seizure identification. We propose a state-of-the-art performing AI system that combines electroencephalogram (EEG) and electrocardiogram (ECG) for seizure identification, tested on clinical data with early evidence demonstrating generalization across hospitals. The model was trained and validated on the publicly available Temple University Hospital (TUH) dataset. To evaluate performance in a clinical setting, we conducted non-patient-specific pseudo-prospective inference tests on three out-of-distribution datasets, including EPILEPSIAE (30 patients) and the Royal Prince Alfred Hospital (RPAH) in Sydney, Australia (31 neurologists-shortlisted patients and 30 randomly selected). Our multimodal approach improves the area under the receiver operating characteristic curve (AUC-ROC) by an average margin of 6.71% and 14.42% for deep learning techniques using EEG-only and ECG-only, respectively. Our model's state-of-the-art performance and robustness to out-of-distribution datasets show the accuracy and efficiency necessary to improve epilepsy diagnoses. To the best of our knowledge, this is the first pseudo-prospective study of an AI system combining EEG and ECG modalities for automatic seizure annotation achieved with fusion of two deep learning networks.