Inter-rater reliability in the diagnosis and classification of epileptic seizures and epilepsy in adults

Accurate classification of epileptic seizures and epilepsy is essential for diagnosis, treatment, and research. This study evaluated inter-rater agreement in seizure and epilepsy diagnosis and classification using standardized clinical vignettes.

Thirty-seven anonymized vignettes, based on adult cases from Merano Hospital (2010-2022), were independently assessed by four epileptologists from different Italian centers. Structured questions covered seizure diagnosis, categorization (acute symptomatic vs. unprovoked), seizure onset, epilepsy diagnosis (ILAE 2014), syndrome identification, etiology, and classification according to the 2017 ILAE basic and expanded systems. Inter-rater agreement was measured with Fleiss' κ and Gwet's AC1; only perfectly matching responses were considered concordant. Bootstrap resampling (1000 iterations) provided 95% confidence intervals.

Agreement was almost perfect for seizure diagnosis (κ = .94, AC1 = .99), epilepsy diagnosis (κ = .96, AC1 = .97), seizure categorization (κ = .86, AC1 = .90), seizure onset (κ = .83, AC1 = .93), and etiology (κ = .84, AC1 = .92). Syndrome identification showed high agreement (κ = .82, AC1 = .97). The ILAE 2017 basic system yielded substantial reliability (κ = .72, AC1 = .82), whereas the expanded version showed lower consistency (κ = .65, AC1 = .77).

This preliminary study, although based on adult emergency cases in one country and rated by only 4 specialists, demonstrates high inter-rater agreement supporting the reliability of current ILAE frameworks. Lower concordance in detailed classifications highlights interpretive variability and the need for refinement. The forthcoming 2025 ILAE updates may improve consistency and diagnostic precision in epilepsy care and research.