Examiner reliability in dental radiography

In long-term investigations involving a large number of study participants, it is frequently necessary to employ the use of multiple examiners who must exhibit high levels of inter- and intra-examiner reliability in order to minimize examiner bias, which can distort scientific findings. This report on the calibration of four examiners in a large project investigating the efficacy of dental radiography shows high levels of examiner reliability using various statistical measures of agreement. Levels of intra-examiner agreement using Cohen's Kappa index were 0.75 and higher at baseline, and remained at approximately the same level (0.80) throughout the 24-month period of the study. The Kappa index of inter-examiner agreement among the six pairings of the four examiners ranged from 0.68 to 0.80 for caries and 0.72 and 0.83 for periodontal disease. Values of four statistical measures of agreement (proportional agreement, Kendall's rank correlation, Cohen's Kappa index, and Cohen's weighted Kappa index) were determined to show the importance of using measures, such as the Kappa index, which take chance agreement into account.