CO-score; a new method for quality assessment of radiology reports

Studies on assessing radiology reports commonly calculates sensitivity, specificity and accuracy, which estimates if the observer has tendency to overdiagnose, overlook pathology, or both. This pilot study examines a new method for assessing the quality of radiology reports, based on the patients' clinical outcome.

Two observers evaluated five hundred reports by four experienced reporting radiographers on X-ray images of the appendicular skeleton. The observers categorised the reports as true or false and gradated the quality of the report from 1 to 3 based on the patients' clinical outcome. We developed a new performance score, called the Consequence of Clinical Outcome (CO-score), which combines the amount of incorrect reports and the severity of errors, to assess the overall quality of the reports. A low CO-score represents high quality with few or inconsiderate errors.

The results showed no direct connection between high accuracy and low CO-score. All radiographers achieved high levels of accuracy (range: 96.8%-100%) but varied in CO-score (range: 0.00-0.14). One radiographer achieved an accuracy of 97.6% but a high CO-score of 0.14 as four reports had clinical consequence for the patients and five reports lacked minor details. One report was classified as true positive but was inadequate and led to wrong treatment.

This study shows that true reports can affect the patients' clinical outcome and reports classified as false can represent insignificant errors. The new CO-score gives a more nuanced view of the reporting quality by including the patients' clinical outcome in the performance score.

We suggest that the CO-score is included as a supplement to the common methods in future studies assessing the quality of radiology reports as well as in clinical audits.