Automated Seizure Classification Using Multimodal Large Language Models

Area of Science:

• Neurology
• Artificial Intelligence
• Medical Imaging Analysis

Background:

• Distinguishing epileptic seizures (ES) from nonepileptic seizures (NES) is clinically challenging, often requiring extensive inpatient video-electroencephalogram (EEG) monitoring.
• Automated analysis of seizure videos could potentially streamline diagnosis and reduce healthcare costs.

Purpose of the Study:

• To develop and evaluate a Multimodal Large Language Models (MLLMs)-based method for automated extraction of semiological features from seizure videos.
• To classify events as ES or NES using the extracted features.

Main Methods:

• A MLLMs framework integrating vision-language models (VLMs) and audio-language models (ALMs) was used to analyze 90 videos of ES and NES events.
• The models automatically extracted 24 clinically relevant semiological features, which were compared to expert annotations.
• Extracted features were used to train classifiers (KNN, XGBoost, Deep Factorization Machine) for ES/NES differentiation using leave-one-patient-out cross-validation.

Main Results:

• Expert-annotated features with KNN achieved high performance (precision 0.97, recall 0.97, F1-score 0.97, AUC 0.99).
• The MLLMs pipeline achieved a mean recall of 0.71, mean accuracy of 0.58, and mean F1-score of 0.51 for feature extraction.
• The best KNN model using MLLMs-extracted features achieved precision 0.77, recall 0.76, F1-score 0.76, and AUC 0.76, correctly identifying 68/90 events.

Conclusions:

• MLLMs can feasibly extract clinically relevant semiological features from seizure videos for automated analysis.
• This MLLMs-based approach offers a promising, clinically interpretable method to assist in diagnosing epilepsy using video recordings.