Radiologists' Self-Assessment Versus Peer Assessment of Perceived Probability of Recommending Additional Imaging

Determine radiologist ability to accurately select the probability of recommendation of additional imaging (RAI) for themselves and colleagues when arrayed in a feedback report.

In this institutional review board-approved study, we analyzed 318,366 diagnostic imaging reports from examinations performed in the radiology department of a large quaternary teaching hospital during calendar year 2016. A validated machine learning algorithm identified reports containing RAI. A multivariable logistic regression model was then used to determine the probability of RAI. In 2018, an e-mailed survey asked radiologists to identify their own RAI probability and that of their colleagues from a report arrayed lowest to highest. Radiologists were grouped into quartiles based on their RAI probability. χ² Analysis compared self-assessment and assessment of colleagues between quartiles.

Forty-eight of 57 radiologists completed the survey (84.2%). Fourteen (29.2%) accurately self-identified their RAI probability (chose the correct quartile); 34 (70.8%) did not. There was no statistically significant difference between quartiles of radiologists and their ability to self-identify their RAI probability (ie, radiologists in the bottom or top quartile of RAI probabilities did not correctly predict their RAI probability). However, radiologists were better able to identify the RAI probability of their colleagues who were in the top and bottom quartiles.

Radiologists were unable to estimate their own RAI probability but were better at predicting the RAI probability of colleagues. Given that radiologists, and physicians in general, may be poor evaluators of their own performance, objective assessment tools are likely needed to help reduce unwarranted variation.