Related Experiment Video
Updated: May 28, 2026
05:37
An R-Based Landscape Validation of a Competing Risk Model
Published on: September 16, 2022
Caution when using prognostic models: a prospective comparison of 3 recent prognostic models.
Antonio Paulo Nassar1, Amilcar Oshiro Mocelin, André Luiz Baptiston Nunes
1Adult Intensive Care Units—Hospital e Maternidade São Camilo—São Paulo, Brazil. paulo_nassar@yahoo.com.br
Journal of Critical Care
|October 29, 2011
Summary
This study evaluated the performance of three general prognostic models in Brazilian intensive care units. While discrimination was good, all models showed poor calibration, highlighting the need for caution when using them for benchmarking.
Area of Science:
- Critical Care Medicine
- Health Services Research
- Biostatistics
Background:
- Prognostic models are crucial for estimating mortality and comparing outcomes across intensive care units (ICUs).
- Validation of these models in diverse populations is essential before widespread clinical application.
- General prognostic models like APACHE IV, SAPS 3, and MPM(0)-III require local validation.
Purpose of the Study:
- To assess the performance of three recently developed general prognostic models: APACHE IV, SAPS 3, and MPM(0)-III.
- To validate these models in a population admitted to Brazilian medical-surgical intensive care units.
- To evaluate the calibration and discrimination capabilities of the selected models in a specific healthcare setting.
Main Methods:
- A cohort of 5780 patients admitted to three Brazilian ICUs between July 2008 and December 2009 was analyzed.
- Model performance was assessed using standardized mortality ratios, the Hosmer-Lemeshow goodness-of-fit test for calibration, and the area under the receiver operator curve (AUC) for discrimination.
- Statistical comparisons were made to determine significant differences in model performance.
Main Results:
- In-hospital mortality was 9.1% in the study population.
- All models demonstrated very good discrimination, with AUC values ranging from 0.840 (MPM(0)-III) to 0.883 (APACHE IV).
- APACHE IV exhibited superior discrimination compared to SAPS 3 and MPM(0)-III (P < .001). However, all models showed poor calibration, significantly overestimating hospital mortality (P < .001 for all).
Conclusions:
- Despite very good discrimination, all evaluated prognostic models exhibited poor calibration in the Brazilian ICU population.
- These findings are consistent with previous validation studies, suggesting a common limitation of general prognostic models.
- Caution is advised when utilizing these prognostic models for benchmarking purposes due to their calibration deficiencies.
More Related Videos
Related Concept Videos
01:20Comparing the Survival Analysis of Two or More Groups
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and Cox...
01:26Pharmacokinetic Models: Comparison and Selection Criterion
Physiological and compartmental models are valuable tools used in studying biological systems. These models rely on differential equations to maintain mass balance within the system, ensuring an accurate representation of the dynamic processes at play.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
01:21Cancer Survival Analysis
Cancer survival analysis focuses on quantifying and interpreting the time from a key starting point, such as diagnosis or the initiation of treatment, to a specific endpoint, such as remission or death. This analysis provides critical insights into treatment effectiveness and factors that influence patient outcomes, helping to shape clinical decisions and guide prognostic evaluations. A cornerstone of oncology research, survival analysis tackles the challenges of skewed, non-normally...
01:17Clearance Models: Noncompartmental Models
Clearance is a pharmacokinetic parameter traditionally defined by compartment models, signifying the rate at which a drug is expelled from the body. However, a noncompartmental model offers an alternative method for assessing clearance, primarily employing empirical data obtained after administering a single drug dose.
The noncompartmental approach capitalizes on extensive sampling data, correlating the volume of distribution to systemic exposure and the administered dosage. This method enables...
The noncompartmental approach capitalizes on extensive sampling data, correlating the volume of distribution to systemic exposure and the administered dosage. This method enables...