Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Sensitivity, Specificity, and Predicted Value01:13

Sensitivity, Specificity, and Predicted Value

1.7K
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
1.7K
Receiver Operating Characteristic Plot01:15

Receiver Operating Characteristic Plot

570
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
570
Data Validation01:03

Data Validation

7.2K
Data validation is an essential part of a comprehensive assessment. Validation is confirming or verifying and opening the door to gathering more assessment data as it clarifies vague or unclear data. The process of checking and verifying the collected information is called data validation. The primary purpose of data validation is to ensure data is as free from error, bias, and misinterpretation as possible.
Nursing assessment guides are generally based on holistic models rather than medical...
7.2K
Prediction Intervals01:03

Prediction Intervals

3.5K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y. 
3.5K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Health-related quality of life profiles and trajectories of patients with post COVID-19 condition: a latent Markov model.

Archives of public health = Archives belges de sante publique·2026
Same author

Assessing Diagnostic Accuracy of a Handheld Point-of-Care Device for Quantifying Neonatal Bilirubin Levels: The BEAT Jaundice @Home Study.

Neonatology·2026
Same author

The impact of an organized screening program on gastric cancer incidence: a quasi-experimental study.

International journal of epidemiology·2026
Same author

Universal Transcutaneous Bilirubin Screening in a Midwifery-Led Home Care Setting.

JAMA network open·2026
Same author

Effects of Causal and Noncausal Components of Screening on Prostate Cancer Mortality: A Mediation Analysis from the Dutch, Finnish, and Swedish Centers of the European Randomized Study of Screening for Prostate Cancer.

European urology oncology·2026
Same author

Acute Carotid Stenting for Tandem Lesions in Patients Randomized to Endovascular Treatment With or Without Thrombolysis: Results From the IRIS Individual Participant Data Meta-Analysis.

Stroke·2025

Related Experiment Video

Updated: Mar 25, 2026

An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.7K

Assessing Discriminative Performance at External Validation of Clinical Prediction Models.

Daan Nieboer1, Tjeerd van der Ploeg2, Ewout W Steyerberg1

  • 1Department of Public Health, Erasmus MC-University medical center, Rotterdam, the Netherlands.

Plos One
|February 17, 2016
PubMed
Summary

A new permutation test for external model validation may mislead when populations differ. Benchmark values are crucial for accurately assessing prediction model transportability and interpreting the c-statistic in new settings.

More Related Videos

Constructing and Visualizing Models using Mime-based Machine-learning Framework
06:19

Constructing and Visualizing Models using Mime-based Machine-learning Framework

Published on: July 22, 2025

3.1K
Identification of Disease-related Spatial Covariance Patterns using Neuroimaging Data
14:27

Identification of Disease-related Spatial Covariance Patterns using Neuroimaging Data

Published on: June 26, 2013

16.5K

Related Experiment Videos

Last Updated: Mar 25, 2026

An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.7K
Constructing and Visualizing Models using Mime-based Machine-learning Framework
06:19

Constructing and Visualizing Models using Mime-based Machine-learning Framework

Published on: July 22, 2025

3.1K
Identification of Disease-related Spatial Covariance Patterns using Neuroimaging Data
14:27

Identification of Disease-related Spatial Covariance Patterns using Neuroimaging Data

Published on: June 26, 2013

16.5K

Area of Science:

  • Biostatistics
  • Health Informatics
  • Epidemiology

Background:

  • External validation is key for assessing prediction model generalizability.
  • A recent permutation test evaluates model transportability using the c-statistic.
  • This study compares this test to existing methods for judging c-statistic changes.

Purpose of the Study:

  • To evaluate a new permutation test for external prediction model validation.
  • To compare its performance against established benchmark values for the c-statistic.
  • To assess model transportability under varying case-mix and predictor effect scenarios.

Main Methods:

  • A simulation study developed logistic regression models on a development set and validated them.
  • Two scenarios were tested: heterogeneous vs. homogeneous case-mix and varying predictor effects.
  • Methods were illustrated using 15 traumatic brain injury datasets.

Main Results:

  • The permutation test showed homogeneity in scenario 1 (heterogeneous case-mix) and heterogeneity in scenario 2 (homogeneous case-mix).
  • Benchmark values accurately identified case-mix differences in both scenarios.
  • The permutation test yielded potentially misleading results regarding population homogeneity.

Conclusions:

  • The permutation test may give misleading results in external validation when case-mix differs.
  • Distinguishing case-mix variations from altered regression coefficients is vital for correct c-statistic interpretation.
  • Existing benchmark methods appear more reliable for assessing prediction model transportability.