Interrater reliability in assessing functional systems and disability on the Kurtzke scale in multiple sclerosis

Interexaminer agreement in the use of quantitative scales for the evaluation of neurological deficits is essential to the reliability of clinical data from cooperative studies on multiple sclerosis. In this study, four neurologists, arranged into six pairs, examined 24 patients with definite multiple sclerosis and assessed each functional system and disability on the Kurtzke scale. As expressed by the kappa index, interobserver agreement was rather low, ranging from 30% to 50%. Sensory and mental functions turned out to be the most variable. The kappa indexes reached values above 85%, when raters who differed by no more than one point were considered as agreeing. A point difference on the scale of, at most, two units seemed to be a reliable index of clinical change. Moreover, these results pointed to the necessity for a specific training program for raters and for periodic control of interobserver variability in multicenter surveys.