The role of clinical "experience" in diagnostic performance

Twenty-one physicians examined records of 43 patients who had attended the hospital because of chest pain. Of these patients, 20 had had coronary heart disease (CHD), 15 had had nonspecific pain, and eight had had pulmonary embolism. The physicians indicated the probability of CHD in each case on the basis of 18 clinical findings, not including ECG, x-ray, or biochemical studies. The trial was repeated five years later, using the same records, by 16 of the same physicians. Diagnostic accuracy was evaluated by ROC curves, and the weight ascribed to each cue was inferred by multiple regression with estimated probability of CHD as the dependent variable. No significant change of areas under the ROC curves with increasing length of clinical experience was observed. Multiple regression was significant in 30 of 37 analyses. The distributions of most physicians' estimates of probabilities had similar shapes five years apart. It is concluded that "experience" does not have a clear role in diagnostic performance based on recorded data and that personal calibration and preferences in estimating probabilities often persist for years.