Variation in restorative treatment decisions: application of Receiver Operating Characteristic curve (ROC) analysis

It has been evident for many years that dentists, when planning treatment for patients, do not act in a standard manner, and previous research has shown there to be wide variations in treatment planning amongst groups of dentists. Signal detection theory and Receiver Operating Characteristic (ROC) analysis allows measurement of an observer's ability to detect a lesion, while at the same time allowing examination of how a lesion, once perceived, is judged to be in need of treatment. An ROC curve is constructed by plotting the sensitivity (or true positive rate) of decisions made, against the false positive rate (equivalent to 1-specificity) when various decision attitudes, from interventionist to non-interventionist, are held. Fifteen pairs of simulated bitewing radiographs were shown to 20 dentists, who were asked to specify, for each approximal lesion, whether or not they would place a conventional restoration. The 7200 decisions made by the dentists were validated by sectioning and microscopically examining the teeth. The mean sensitivity of the dentists' decisions, when the strictest operating thresholds were held and caries into dentine was the validating criterion, was 0.26 and the mean specificity was 0.96. ROC analysis shows that when operating at the strictest threshold, the dentists were implying that specificity was weighted as being 2.7 times more important than sensitivity. ROC analysis leads to insight into how dentists differentially weight the true and false, positive and negative, outcomes of their decisions and thus allows explanation of why two dentists would rarely make exactly the same treatment plan for one patient, and also why different treatments might be offered to two patients exhibiting the same levels of disease.