Reducing variability in treatment decision-making: effectiveness of educating clinicians about uncertainty

The objective of this study was to investigate whether or not education about the concept of uncertainty reduced variability in treatment decision-making. Three small groups of dentists in North York, Canada were asked to make restorative treatment decisions about simulated bitewing radiographs. They subsequently took part in a seminar about variations in perception and judgement and were given explanations of sensitivity, specificity and receiver operating characteristic (ROC) curve analysis. A repeat reading of the radiographs was then performed by both test and control groups. Results indicated that the intervention increased the accuracy, and decreased the variability of dentists' restorative treatment decisions. Kappa statistics were 0.33, 0.34 and 0.31 before the seminar, and 0.40, 0.43 and 0.41 after the seminar. Standard errors for kappas were 0.06, 0.05 and 0.05 before the seminar, and 0.02, 0.02 and 0.05 after the seminar. The area under the ROC curve was 0.7136 before the seminar and 0.7835 after the seminar. The data demonstrate that the dentists' decisions were less variable and more accurate following the educative intervention. This study suggests that there is potential for improving consistency and accuracy in clinical decision-making through education in probabilistic reasoning.