Interobserver Agreement of Lung Ultrasound Findings of COVID-19

Lung ultrasound (LUS) has received considerable interest in the clinical evaluation of patients with COVID-19. Previously described LUS manifestations for COVID-19 include B-lines, consolidations, and pleural thickening. The interrater reliability (IRR) of these findings for COVID-19 is unknown.

This study was conducted between March and June 2020. Nine physicians (hospitalists: n = 4; emergency medicine: n = 5) from 3 medical centers independently evaluated n = 20 LUS scans (n = 180 independent observations) collected from patients with COVID-19, diagnosed via RT-PCR. These studies were randomly selected from an image database consisting of COVID-19 patients evaluated in the emergency department with portable ultrasound devices. Physicians were blinded to any patient information or previous LUS interpretation. Kappa values (κ) were used to calculate IRR.

There was substantial IRR on the following items: normal LUS scan (κ = 0.79 [95% CI: 0.72-0.87]), presence of B-lines (κ = 0.79 [95% CI: 0.72-0.87]), ≥3 B-lines observed (κ = 0.72 [95% CI: 0.64-0.79]). Moderate IRR was observed for the presence of any consolidation (κ = 0.57 [95% CI: 0.50-0.64]), subpleural consolidation (κ = 0.49 [95% CI: 0.42-0.56]), and presence of effusion (κ = 0.49 [95% CI: 0.41-0.56]). Fair IRR was observed for pleural thickening (κ = 0.23 [95% CI: 0.15-0.30]).

Many LUS manifestations for COVID-19 appear to have moderate to substantial IRR across providers from multiple specialties utilizing differing portable devices. The most reliable LUS findings with COVID-19 may include the presence/count of B-lines or determining if a scan is normal. Clinical protocols for LUS with COVID-19 may require additional observers for the confirmation of less reliable findings such as consolidations.