Quantifying the impact of a computer-aided diagnostic score on the clinical diagnosis of functional seizures

The diagnosis of functional/dissociative seizures (FDS) without ictal video-electroencephalography is challenging. The Functional/Dissociative Seizures Likelihood Score (FSLS) is a machine learning-based diagnostic score that aims to help clinicians identify FDS. We evaluated whether a human-in-the-loop implementation of the FSLS improved the performance of clinicians identifying FDS as compared to epileptic seizures (ES).

We constructed 117 anonymized cases about patients with ictal video-electroencephalography-documented FDS, epilepsy, co-occurring ES and FDS, or physiological seizurelike events. Text-based clinical history was presented followed by the FSLS. Readers were asked the most likely diagnosis after each piece of information. We used mixture modeling combined with mixed effects logistic regression to perform data-driven grouping of participants based on observed patterns of diagnostic performance.

Overall, 163 readers saw 1142 cases (median = 4 cases/reader), and 146 (90%) had a performance higher than chance. More formal training in seizures was associated with better performance (epileptologist accuracy = 67%, mental health clinician accuracy = 52%). Data-driven groups including 66% of readers benefitted from the FSLS (accuracy improvement = 12%-15%, p < .05), including those in the reference and near highest baseline performance group. Other groups had no net change in performance (p > .75).

Clinicians with more formal seizure training identified possible FDS more accurately than others, but formal training did not guarantee high diagnostic performance. Two performance-based groups, which included 66% readers, benefitted from the FSLS because they identified when to change their mind on the basis of the FSLS's suggestion. The implementation of machine learning in the diagnosis of FDS should focus on identifying clinical settings where it can effectively enhance clinicians' decision-making.