Impact of the validator and the validation method on the outcome of occlusal caries diagnosis

In studies evaluating the performance of caries-diagnostic methods, a validation of the true state of disease is needed. The aim of this study was to evaluate the impact of the validator and the validation system (stereomicroscopy or radiography of tooth sections) on the outcome of diagnostic tests for occlusal caries. The material consisted of 60 extracted third molars which were serially sectioned (500-600 microns thick). Four observers examined the sections by two caries validation methods: stereomicroscopy and film radiography. The presence of caries in the occlusal surfaces of these teeth had previously been recorded by visual inspection and conventional film radiography. The kappa values for interobserver agreement within one validation method ranged from 0.44 to 0.76 for radiography and from 0.47 to 0.60 for microscopy. The intraobserver agreement with the two methods was low (range 0.31-0.49), and by cross-tabulating the data, it was found that the disagreements originated in a consistently deeper lesion score with stereomicroscopy than with radiography by all observers except 1. By use of receiver operating characteristic curve areas, little impact of the validation method was seen when visual inspection was validated (against microscopy mean area = 0.75, against radiography mean area = 0.74). The mean receiver operating characteristic area was higher when diagnostic film radiographs were validated against radiography (0.68) than against microscopy (0.63). The differences between observers within a validation method were larger with microscopy than with radiography. In conclusion, caries validation methods are subject to variability. The outcome of caries--diagnostic tests may be influenced both by the validator and the validation method.