Related Experiment Video
Updated: Sep 26, 2025
Lung CT Segmentation to Identify Consolidations and Ground Glass Areas for Quantitative Assesment of SARS-CoV Pneumonia
Published on: December 19, 2020
COVID-19 pneumonia chest radiographic severity score: variability assessment among experienced and in-training
Marly van Assen1, Mohammadreza Zandehshahvar2, Hossein Maleki2
1Department of Radiology and Imaging Sciences, Emory University Hospital | Emory Healthcare, Inc., Atlanta, GA, USA.
Objective:
The purpose was to evaluate reader variability between experienced and in-training radiologists of COVID-19 pneumonia severity on chest radiograph (CXR), and to create a multireader database suitable for AI development.
Methods:
In this study, CXRs from polymerase chain reaction positive COVID-19 patients were reviewed. Six experienced cardiothoracic radiologists and two residents classified each CXR according to severity. One radiologist performed the classification twice to assess intraobserver variability. Severity classification was assessed using a 4-class system: normal (0), mild (1), moderate (2), and severe (3). A median severity score (Rad Med) for each CXR was determined for the six radiologists for development of a multireader database (XCOMS). Kendal Tau correlation and percentage of disagreement were calculated to assess variability.
Results:
A total of 397 patients (1208 CXRs) were included (mean age, 60 years SD ± 1), 189 men). Interobserver variability between the radiologists ranges between 0.67 and 0.78. Compared to the Rad Med score, the radiologists show good correlation between 0.79-0.88. Residents show slightly lower interobserver agreement of 0.66 with each other and between 0.69 and 0.71 with experienced radiologists. Intraobserver agreement was high with a correlation coefficient of 0.77. In 220 (18%), 707 (59%), 259 (21%) and 22 (2%) CXRs there was a 0, 1, 2 or 3 class-difference. In 594 (50%) CXRs the median scores of the residents and the radiologists were similar, in 578 (48%) and 36 (3%) CXRs there was a 1 and 2 class-difference.
Conclusion:
Experienced and in-training radiologists demonstrate good inter- and intraobserver agreement in COVID-19 pneumonia severity classification. A higher percentage of disagreement was observed in moderate cases, which may affect training of AI algorithms.
Advances In Knowledge:
Most AI algorithms are trained on data labeled by a single expert. This study shows that for COVID-19 X-ray severity classification there is significant variability and disagreement between radiologist and between residents.
More Related Videos
07:53Author Spotlight: Advancing 3D Modeling for Enhanced Diagnosis and Treatment of Pulmonary Nodules in Early-Stage Lung Cancer
Published on: October 13, 2023
02:09Multi-modal Pulmonary Imaging: Using Complementary Information from CT and Hyperpolarized 129Xe MRI to Evaluate Lung Structure-Function
Published on: April 12, 2024
Related Concept Videos
01:30Radiological Investigation II: MRI and Ventilation Perfusion Scan
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
01:25Imaging Studies for Cardiovascular System VI: Calcium -Scoring CT
01:20Imaging Studies for Cardiovascular System III: X-Ray
Definition and Purpose
An X-ray, or radiograph, is a non-invasive method that uses ionizing radiation to take images of internal structures. It is mainly used in cardiac imaging to examine the heart, lungs, and major blood vessels, aiming to identify abnormalities in the heart's size, shape, and position, such as heart failure, congenital defects, and vascular...
01:13Radiological Investigation III: Pulmonary Angiogram and PET Scan
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
01:30Pneumonia III: Complications and Assessment
01:27Chronic Obstructive Pulmonary Disease-IV: Assessement and Diagnostic Studies
Medical History