Comparison of rheumatological diagnoses by a Bayesian program and by physicians

A Bayesian decision support system was developed for the diagnosis of rheumatic disorders. Knowledge in this system is represented as evidential weights of findings. Simple weights were calculated as the logarithm of likelihood ratios on the basis of 1,000 consecutive patients from a rheumatological clinic. The effect of various methods to improve performance of the system by modification of the weights was studied. Three methods had a mathematical basis; a fourth consisted of weights adapted by a human expert, which allowed inclusion of diagnostic rules such as defined in widely accepted criteria sets. The system's performance was measured in a test population of 570 different cases from the same clinic and compared with predictions of diagnostic outcome made by rheumatologists. The weights from a human expert gave optimal results (sensitivity 65% and specificity 96%), that were close to the physicians' predictions (sensitivity 64% and specificity 98%). The methods to measure the performance of the various models used in this study emphasize sensitivity, specificity and the use of receiver operating characteristics.