Diagnostic performance and interobserver agreement of CO-RADS: evaluation of classification in radiology practice

We aimed to evaluate the use of the COVID-19 reporting and data system (CO-RADS) among radiologists and the diagnostic performance of this system.

Four radiologists retrospectively evaluated the chest CT examinations of 178 patients. The study included 143 patients with positive reverse transcriptase-polymerase chain reaction (RT-PCR) test results and 35 patients whose RT-PCR tests were negative but whose clinical and/or radiological findings were consistent with COVID-19. Fleiss' kappa (κ) values were calculated, and individual observers' scores were compared. To investigate diagnostic efficiency, receiver operating characteristic (ROC) curves were calculated for each interpreter.

The interpreters were in full agreement on 574 of 712 (80.6%) evaluations. The common Fleiss' κ value of all the radiologists combined was 0.712 (95% confidence interval [CI] 0.692-0.769). A reliable prediction on the basis of RT-PCR and clinical findings indicated the mean area under the curve (AUC) of Fleiss' κ value as 0.89 (95% CI 0.708-0.990). General interpreter agreement was found to range from moderate to good.

The interpreter agreement for CO-RADS categories 1 and 5 was reasonably good. We conclude that this scoring system will make a valuable contribution to efforts in COVID-19 diagnosis. CO-RADS can also be of significant value for the diagnosis and treatment of the disease in cases with false-negative PCR results.