The association between operator confidence and accuracy of ultrasonography performed by novice emergency physicians

The variable accuracy of emergency department (ED) ultrasound described in the literature has limited its utility as the sole imaging modality in critical decision making. Although ultrasound accuracy is highly dependent upon the technical abilities of the operator and conditions unique to each patient, no previous study of ED ultrasound has included estimates of operator confidence. This prospective observational study explores the association between operator confidence and the accuracy of ED ultrasound. Ultrasound was not performed in our ED until a formal training module was instituted. Patients were enrolled prospectively for the first year following the training module if they underwent one of the following ultrasound studies: abdominal examination for intraperitoneal fluid, right upper quadrant examination for gallstones, renal examination for hydronephrosis, pelvic examination for intrauterine pregnancy, abdominal examination for aorta diameter > 3 cm, or cardiac examination for pericardial fluid. In addition, formal ultrasound, computed tomography, magnetic resonance imaging, or an invasive procedure was required as a "gold standard" for each patient. Operators recorded their interpretation of the ED ultrasound and rated their confidence with the analysis before the formal imaging study or procedure. Test performance characteristics for each examination type and for all studies together were determined. The association between operator confidence and accuracy was explored using logistic regression and by determining test performance characteristics with patients stratified by confidence value. A total of 276 ED ultrasound studies were included. There were no significant differences in accuracy between ED attendings and residents. Overall accuracy, sensitivity, specificity, LR+, and LR- were 90%, 92%, 86%, 6.8, and 0.09, respectively. With confidence scores of 9 or 10 (n = 113), these values improved to 96%, 99%, 90%, 9.6, and 0.01, respectively. Logistic regression revealed an association between confidence and ED ultrasound accuracy (p < 0.001). It is concluded that a significant association exists between operator confidence and the accuracy of ED ultrasound. High confidence values are associated with clinically useful test performance characteristics.