Inter-observer Variability in the Analysis of CO-RADS Classification for COVID-19 Patients

During the early stages of the pandemic, computed tomography (CT) of the chest, along with serological and clinical data, was frequently utilized in diagnosing COVID-19, particularly in regions facing challenges such as shortages of PCR kits. In these circumstances, CT scans played a crucial role in diagnosing COVID-19 and guiding patient management. The COVID-19 Reporting and Data System (CO-RADS) was established as a standardized reporting system for cases of COVID-19 pneumonia. Its implementation necessitates a high level of agreement among observers to prevent any potential confusion. This study aimed to assess the inter-observer agreement between physicians from different specialties with variable levels of experience in their CO-RADS scoring of CT chests for confirmed COVID-19 patients, and to assess the feasibility of applying this reporting system to those having little experience with it. All chest CT images of patients with positive RT-PCR tests for COVID-19 were retrospectively reviewed by seven observers. The observers were divided into three groups according to their type of specialty (three radiologists, three house officers, and one pulmonologist). The observers assessed each image and categorized the patients into five CO-RADS groups. A total of 630 participants were included in this study. The inter-observer agreement was almost perfect among the radiologists, substantial among a pulmonologist and the house officers, and moderate-to-substantial among the radiologists, the pulmonologist, and the house officers. There was substantial to almost perfect inter-observer agreement when reporting using the CO-RADS among observers with different experience levels. Although the inter-observer variability among the radiologists was high, it decreased compared to the pulmonologist and house officers. Radiologists, house officers, and pulmonologists applying the CO-RADS can accurately and promptly identify typical CT imaging features of lung involvement in COVID-19.