Evaluation of the diagnostic performance of the expert EMG assistant MUNIN

The diagnostic performance of the medical expert system MUNIN for diagnosis of neuromuscular disorders was evaluated on a set of 30 test cases. The cases were provided by 7 experienced electromyographers who were subsequently invited to participate in the evaluation. To reasonably cover the range of disorders, the electromyographers were asked to provide cases from patients with different types of muscular dystrophy, with neuromuscular transmission disorders, with motor neurone disease, and with different types of polyneuropathies. In addition, patients with a range of local neuropathies were provided. Out of the 30 cases, 11 cases were evaluated by an "almost peer review" method and the remaining 19 cases were evaluated by a "silver standard" method. The number of cases evaluated by "almost peer review" was limited to 11 due to time constraints on the evaluation procedure. During the "almost peer review," each electromyographer was asked to diagnose patients, using a vocabulary that closely resembled MUNIN's vocabulary. Subsequently, we attempted to provide a consensus diagnosis for the patients based on discussion among the participating electromyographers. The electromyographers were also asked to assess how well MUNIN had performed in each case. The remaining 19 cases were evaluated by a "silver standard" procedure, where MUNIN's diagnosis was compared to the diagnosis of the expert who provided the case. The results indicated that MUNIN performed well, and the electromyographers considered "that MUNIN performed at the same level as an experienced neurophysiologist." In particular, it was noted that MUNIN handled cases with conflicting findings well, and that it was able to diagnose patients with multiple diseases.