Interrater reliability of Alzheimer's disease diagnosis

To determine interrater reliability of dementia diagnosis, 4 physicians experienced in the evaluation of dementia patients applied 3 sets of diagnostic criteria to each of 62 patients, based on a standardized set of medical record information. All patients had undergone similar examinations and follow-up to establish the initial clinical diagnosis (76% had autopsy). Raters were blind to the diagnosis and to follow-up information after the initial evaluation period. This paper presents interrater agreement (kappa values) for a diagnosis of Alzheimer's disease using the American Psychiatric Association diagnostic criteria from the Diagnostic and Statistical Manual (DSM-III), the National Institute of Neurological and Communicative Disorders and Stroke (NINCDS) criteria for the clinical diagnosis of Alzheimer's disease, and the Eisdorfer and Cohen Research Diagnostic Criteria (ECRDC) for primary neuronal degeneration. The NINCDS showed somewhat higher average interrater reliability (kappa = 0.64) than the DSM-III (kappa = 0.55) and considerably higher interrater reliability than the ECRDC (kappa = 0.37). One rater displayed conspicuously lower levels of interrater reliability than the other 3, especially in DSM-III and ECRDC. This study indicates that interrater reliability of DSM-III and NINCDS criteria are comparable. Documentation of interrater reliability and, if necessary, training to improve reliability is an important consideration in research where different observers are diagnosing dementing illnesses.