Related Experiment Video
Updated: Jun 9, 2026

An R-Based Landscape Validation of a Competing Risk Model
Published on: September 16, 2022
External validity of risk models: Use of benchmark values to disentangle a case-mix effect from incorrect
Yvonne Vergouwe1, Karel G M Moons, Ewout W Steyerberg
1Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands. y.vergouwe@umcutrecht.nl
Abstract:
Various performance measures related to calibration and discrimination are available for the assessment of risk models. When the validity of a risk model is assessed in a new population, estimates of the model's performance can be influenced in several ways. The regression coefficients can be incorrect, which indeed results in an invalid model. However, the distribution of patient characteristics (case mix) may also influence the performance of the model. Here the authors consider a number of typical situations that can be encountered in external validation studies. Theoretical relations between differences in development and validation samples and performance measures are studied by simulation. Benchmark values for the performance measures are proposed to disentangle a case-mix effect from incorrect regression coefficients, when interpreting the model's estimated performance in validation samples. The authors demonstrate the use of the benchmark values using data on traumatic brain injury obtained from the International Tirilazad Trial and the North American Tirilazad Trial (1991-1994).
Related Concept Videos
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Mechanistic Models: Compartment Models in Individual and Population Analysis
Bias in Epidemiological Studies
Strategies for Assessing and Addressing Confounding
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
Regression Toward the Mean
Confounding in Epidemiological Studies
