Effects of specific criteria and calibration on examiner reliability

The purpose of this pilot study was to investigate the use of specific criteria and examiner calibration on the reliability of inexperienced examiners on dental sealant evaluations. Dental (N = 8) and dental hygiene (N = 8) students participated as examiners. The study objectives were to identify differences in calibrated and non-calibrated examiners, examiners calibrated by an expert or non-expert, and reliability between dental and dental hygiene student examiners. A criterion-referenced evaluation form was used to evaluate dental sealant end product on 20 teeth, twice by each examiner. Eight of 16 examiners participated in a one-hour calibration session between evaluations. The session consisted of a discussion of operational definitions, the evaluation procedure for dental sealants, and use of the criterion-referenced form. Intra- and interexaminer reliabilities were measured. There were no statistically significant differences (p less than .05) in intraexaminer reliability. Although calibration produced no significant increase in interexaminer reliability, the post-training reliability scores for the group calibrated by an expert decreased, and scores for the group calibrated by a non-expert increased. No significant difference was found in reliability between dental and dental hygiene student examiners.