Reliability of the El Escorial diagnostic criteria for amyotrophic lateral sclerosis

The El Escorial diagnostic criteria are the most commonly used in clinical studies and therapeutic trials in patients with amyotrophic lateral sclerosis (ALS). The accuracy of the El Escorial criteria was tested in clinical practice, but the reliability is unknown when the diagnosis of ALS must be assessed on the basis of medical records.

To assess the reliability of the El Escorial criteria for the diagnosis of ALS in different settings.

Semistructured forms were used to include the main diagnostic information on 20 patients with definite (n = 6), probable (n = 6), possible (n = 6), and suspected ALS (n = 2) and 19 patients with clinical conditions considered in the differential diagnosis. Agreement was tested by comparing the diagnosis made by the attending physician (the 'gold standard') with that of 4 raters with different backgrounds: a teaching neurologist with research and practical experience in the field of motor neuron disorders, a neurologist with specific interest in motor neuron disorders and neurophysiological background, a neurophysiologist, and a general neurologist with only occasional ALS patients. Sources of disagreement were discussed and the agreement was tested further on the medical records of 98 additional cases taken from an ongoing ALS registry. Eight additional cases (ALS: 4; other conditions: 4) were examined directly by the 4 raters.

The interrater agreement on the medical records was poor (overall kappa 0.05-0.29). When the differential diagnosis was made between ALS (all diagnostic levels) and other conditions, interrater agreement was at best modest, with moderate variations when raters were compared in pairs (kappa 0.03-0.58) and when each rater was compared with the physician (kappa 0.27-0.51). Agreement was higher after direct examination of the patients (kappa 0.33-1) and increased significantly on the medical records after training (overall kappa 0.52-0.79). However, concordance was low (overall kappa 0.08-0.36), even after training, at the lowest diagnostic level (definite to suspected ALS vs. other conditions).

The El Escorial criteria are a poor diagnostic indicator when patients' records are examined. Although medical education significantly improves the reliability of the criteria, concordance is still modest when the diagnosis includes suspected ALS.