Towards trustworthy seizure onset detection using workflow notes

Area of Science:

• Artificial Intelligence in Healthcare
• Medical Signal Processing
• Machine Learning for Clinical Applications

Background:

• Trustworthiness is a critical barrier to deploying healthcare AI, with model robustness across subgroups being a key concern.
• Existing AI models for seizure onset detection often rely on non-causal features, leading to performance degradation in hidden subgroups.
• Routine clinical workflow notes, containing diverse event descriptions, offer a valuable, underutilized data source for AI model training.

Purpose of the Study:

• To improve the trustworthiness and robustness of AI models for seizure onset detection using clinical workflow notes.
• To evaluate the performance gains from scaling training data and employing a multilabel classification approach.
• To address subgroup performance disparities and reduce false positive rates in electroencephalogram (EEG) analysis.

Main Methods:

• Utilized 68,920 hours of EEG data annotated with routine clinical workflow notes, including seizure and non-seizure event descriptions.
• Developed and compared a binary seizure onset detection model with a multilabel model classifying 26 additional attributes (e.g., artifacts).
• Assessed model performance using Area Under the Receiver Operating Characteristic (AUROC) curves and False Positive Rates (FPR) across various subgroups and conditions.

Main Results:

• Scaling training data with workflow notes improved seizure onset detection by 12.3 AUROC points compared to smaller, gold-standard datasets.
• The initial binary model showed performance gaps across subgroups (up to 6.5 AUROC points) and higher FPRs on non-epileptiform abnormalities (+19 FPR points).
• The multilabel model enhanced overall performance (+5.9 AUROC points), improved subgroup performance (up to +8.3 AUROC points), and reduced false positives on non-epileptiform abnormalities (by 8 FPR points).

Conclusions:

• Clinical workflow notes are a powerful resource for training robust and trustworthy healthcare AI, significantly improving seizure onset detection.
• A multilabel approach effectively addresses model biases and enhances performance across diverse patient subgroups and EEG signal characteristics.
• The developed multilabel model demonstrated a twofold improvement in clinical utility, reducing false positives per 24 EEG hours.