Related Experiment Video
Updated: Aug 23, 2025

An R-Based Landscape Validation of a Competing Risk Model
Published on: September 16, 2022
Predicting COVID-19 prognosis in the ICU remained challenging: external validation in a multinational regional cohort
Daniek A M Meijs1, Sander M J van Kuijk2, Laure Wynants3
1Department of Intensive Care Medicine, Maastricht University Medical Centre (Maastricht UMC+), Maastricht, The Netherlands; Department of Intensive Care Medicine, Laurentius Ziekenhuis, Roermond, The Netherlands; Cardiovascular Research Institute Maastricht (CARIM), Maastricht, The Netherlands.
Insights
External validation of COVID-19 prognostic models in the Euregio Intensive Care COVID cohort revealed that only two of nine models demonstrated adequate performance. Better models are needed for future pandemics to aid ICU admission decisions.
Area of Science:
- Critical Care Medicine
- Epidemiology
- Health Informatics
Background:
- Numerous prognostic models for coronavirus disease 2019 (COVID-19) have been developed.
- External validation is crucial before implementing these models in intensive care units (ICUs).
Conclusions:
- Only two of the nine externally validated prognostic models exhibited reasonable discrimination and moderate calibration in the EICC cohort.
- There is a need for improved prognostic models utilizing routine data to support ICU admission decisions during future pandemics.
Objectives:
Many prediction models for coronavirus disease 2019 (COVID-19) have been developed. External validation is mandatory before implementation in the intensive care unit (ICU). We selected and validated prognostic models in the Euregio Intensive Care COVID (EICC) cohort.
Study Design And Setting:
In this multinational cohort study, routine data from COVID-19 patients admitted to ICUs within the Euregio Meuse-Rhine were collected from March to August 2020. COVID-19 models were selected based on model type, predictors, outcomes, and reporting. Furthermore, general ICU scores were assessed. Discrimination was assessed by area under the receiver operating characteristic curves (AUCs) and calibration by calibration-in-the-large and calibration plots. A random-effects meta-analysis was used to pool results.
Results:
551 patients were admitted. Mean age was 65.4 ± 11.2 years, 29% were female, and ICU mortality was 36%. Nine out of 238 published models were externally validated. Pooled AUCs were between 0.53 and 0.70 and calibration-in-the-large between -9% and 6%. Calibration plots showed generally poor but, for the 4C Mortality score and Spanish Society of Infectious Diseases and Clinical Microbiology (SEIMC) score, moderate calibration.
Conclusion:
Of the nine prognostic models that were externally validated in the EICC cohort, only two showed reasonable discrimination and moderate calibration. For future pandemics, better models based on routine data are needed to support admission decision-making.
Related Concept Videos
Prediction Intervals
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
Cancer Survival Analysis
Statistical Methods for Analyzing Epidemiological Data
Steps in Outbreak Investigation
Comparing the Survival Analysis of Two or More Groups
Sensitivity, Specificity, and Predicted Value
Sensitivity is the...

