Grading of tenderness as a source of interrater error in the Ritchie articular index

To assess the effect of grading of tenderness on the interrater reliability of the Ritchie articular index (RAI), 3 physicians recorded independent joint scores on each of 18 patients, examined in random order. Our results indicate that close agreement (Intraclass Correlation Coefficient = 0.85) can be achieved on global RAI scores, and that raters can achieve reasonable agreement (kappa = 0.40-0.59) on the absolute presence or absence of tenderness of individual joints. By contrast, interrater agreement hardly exceeds the chance level (kappa = 0.008-0.148) when degree of tenderness is independently assessed. The grading system of the RAI may thus be implicated as an important source of the instrument's interrater error.