Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

External validation is necessary in prediction research: a clinical example.

S E Bleeker1, H A Moll, E W Steyerberg

  • 1Erasmus Medical Center/Sophia Children's Hospital Department of Pediatrics, Room Sp 1545 Dr Molewaterplein 60, 3015 GJ Rotterdam, The Netherlands. sebleeker@yahoo.com

Journal of Clinical Epidemiology
|September 25, 2003
PubMed
Summary

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Changes in participatory and societal outcomes during the waiting period for cochlear implantation - an observational study.

European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery·2024
Same author

Consolidation and prediction of long-term treatment effect of group and online mindfulness-based cognitive therapy for distressed cancer patients.

Acta oncologica (Stockholm, Sweden)·2018
Same author

Subarachnoid Hemorrhage in Germany Between 2010 and 2013: Estimated Incidence Rates Based on a Nationwide Hospital Discharge Registry.

World neurosurgery·2017
Same author

Mindfulness-based stress reduction added to care as usual for lung cancer patients and/or their partners: A multicentre randomized controlled trial.

Psycho-oncology·2017
Same author

Cognitive reactivity as outcome and working mechanism of mindfulness-based cognitive therapy for recurrently depressed patients in remission.

Cognition & emotion·2017
Same author

Predictors of perceived stigmatization in patients with psoriasis.

The British journal of dermatology·2016

Internal validation using bootstrapping may overestimate diagnostic prediction model performance. External validation is crucial for assessing generalizability in new patient groups, especially with smaller datasets.

Area of Science:

  • Medical Informatics
  • Clinical Epidemiology
  • Biostatistics

Background:

  • Prediction models often show optimistic performance on derivation data.
  • Internal validation, including bootstrapping, aims to correct for this optimism.
  • However, internal validation may not adequately assess generalizability to new settings.

Purpose of the Study:

  • To illustrate the limitations of internal validation for diagnostic prediction models.
  • To emphasize the necessity of external validation for determining model generalizability.
  • To evaluate a prediction model for serious bacterial infections in febrile children.

Main Methods:

  • A diagnostic prediction model was developed for serious bacterial infections in children.
  • Internal validation was performed using bootstrap resampling techniques.

Related Experiment Videos

  • External validation was conducted on a separate dataset.

    • Main Results:

    • The model's apparent performance in the derivation set (n=376) was high (AUC 0.83), improving to 0.76 after bootstrap correction.
    • Performance significantly dropped in the external validation set (n=179) with an AUC of 0.57.
    • This highlights a substantial performance decline upon external validation.

    Conclusions:

    • Bootstrap-corrected internal validation may be insufficient for small datasets.
    • External validation is essential to confirm a prediction model's performance in real-world clinical practice.
    • Implementing prediction models requires robust external validation to ensure reliability.