Identifying COVID-19 Disease Severity in Real-World Data: Implications for Medical Product Effectiveness Studies

The U.S. Food and Drug Administration (FDA) defined disease severity criteria to assist clinical development of medical products for management of COVID-19. These definitions were translated to code-based algorithms for use in real-world data. We validated the algorithms' performance in ambulatory settings at three regional integrated healthcare delivery systems contributing data to FDA's Sentinel System.

We identified cohorts of individuals ≥ 18 years that met the algorithms' criteria for mild, moderate, and severe COVID-19 at incident COVID-19 diagnosis or positive SARS-CoV-2 test, and separately, at incident COVID-19 treatment, from January 2022 through April 2023. We validated the algorithms via chart review of a random sample to calculate positive predictive values (PPVs) and 95% CIs.

The algorithms identified 33 071 patients at COVID-19 diagnosis or positive test; 26 985 mild (49 chart reviewed), 5180 moderate (55 reviewed), and 906 severe (56 reviewed). A total of 4512 patients were identified at COVID-19 treatment; 3474 mild (56 reviewed), 848 moderate (60 reviewed), and 190 severe (46 reviewed). The PPVs (1) at COVID-19 diagnosis or positive test were: mild 57% (95% CI: 43%-71%), moderate 58% (95% CI: 45%-71%), and severe 54% (95% CI: 41%-67%), and (2) at COVID-19 treatment: mild 57% (95% CI: 44%-70%), moderate 70% (95% CI: 58%-82%) and severe 72% (95% CI: 59%-85%).

The algorithms had low-to-moderate performance in classifying COVID-19 severity in ambulatory settings, depending on assessment at diagnosis or treatment. Researchers should consider the performance of the algorithm when using real-world data to assess COVID-19 severity.