A rater training program in removable prosthodontics

Four faculty members in removable prosthodontics participated in a three-phase rater training program. Nine complete dentures were given an overall rating by three outside experts to obtain an accuracy measure. A three-point rating scale was used: R (cannot be appreciably improved), S (clinically acceptable), and T (clinically unacceptable). During the pretraining phase, the average interrater reliability was .57 as estimated by intraclass correlation, and the mean accuracy correlation was .76. The rater training phase consisted of a four-hour session including presentations and discussions of rating terminology and formats, observer accuracy, and related issues. In the post-training phase, the same nine complete dentures were rated by the four faculty members, using the same three-point rating scale and a five-point scale. The average post-training interrater reliabilities were .56 and .76 on the three- and five-point scales, respectively. The mean accuracy correlation was .78.