Inter-Rater Reliability of EEG-Based Encephalopathy Grading

Area of Science:

• Neurology
• Neurophysiology
• Medical Informatics

Background:

• Encephalopathy severity assessment is crucial for patient management.
• The Visual EEG Confusion Assessment Method-Severity (VE-CAM-S) is a tool to quantify encephalopathy based on electroencephalography (EEG) features.
• Evaluating the reliability of VE-CAM-S among experts is essential for its clinical adoption.

Purpose of the Study:

• To assess the inter-rater reliability of the VE-CAM-S scale among neurological experts.
• To evaluate the performance of experts in identifying specific EEG features used in VE-CAM-S scoring.
• To identify common errors in feature recognition that impact VE-CAM-S scores.

Main Methods:

• Nine experts independently reviewed 32 EEG samples, scoring 29 features (VE-CAM-S and additional).
• A consensus panel of three experts established the gold standard for scoring.
• Performance was measured using the Matthews correlation coefficient, sensitivity, and specificity, with qualitative analysis of errors.

Main Results:

• Experts achieved a median Matthews correlation coefficient of 0.82, indicating good reliability.
• High specificity (>90%) was observed for most features, but sensitivity varied, with lower scores for burst suppression, EDB, and background rhythms.
• Common errors included missing subtle findings and misidentifying features like extreme delta brush.

Conclusions:

• The VE-CAM-S scale demonstrates initial support for reliability in grading encephalopathy.
• Errors were most significant for high-weight features, suggesting areas for improvement.
• Recommendations include refining feature definitions, creating visual aids, and enhancing educational materials for better accuracy.