Implementation of the Radiological Society of North America Expert Consensus Guidelines on Reporting Chest CT...

RSNA expert consensus guidelines provide a framework for reporting CT findings related to COVID-19, but have had limited multireader validation.

To assess the performance of the RSNA guidelines and quantify interobserver variability in application of the guidelines in patients undergoing chest CT for suspected COVID-19 pneumonia.

A retrospective search from 1/15/20 to 3/30/20 identified 89 consecutive CT scans whose radiological report mentioned COVID-19. One positive or two negative RT-PCR tests for COVID-19 were considered the gold standard for diagnosis. Each chest CT scan was evaluated using RSNA guidelines by 9 readers (6 fellowship trained thoracic radiologists and 3 radiology resident trainees). Clinical information was obtained from the electronic medical record.

There was strong concordance of findings between radiology training levels with agreement ranging from 60 to 86% among attendings and trainees (kappa 0.43 to 0.86). Sensitivity and specificity of "typical" CT findings for COVID-19 per the RSNA guidelines were on average 86% (range 72%-94%) and 80.2% (range 75-93%), respectively. Combined "typical" and "indeterminate" findings had a sensitivity of 97.5% (range 94-100%) and specificity of 54.7% (range 37-62%). A total of 163 disagreements were seen out of 801 observations (79.6% total agreement). Uncertainty in classification primarily derived from difficulty in ascertaining peripheral distribution, multiple dominant disease processes, or minimal disease.

The "typical appearance" category for COVID-19 CT reporting has an average sensitivity of 86% and specificity rate of 80%. There is reasonable interreader agreement and good reproducibility across various levels of experience.