Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Data Validation

Data Validation

Method validation is a crucial process in analytical chemistry designed to confirm that a given method consistently produces reliable and high-quality results. This process is essential when a method is applied to different sample matrices or when procedural modifications are made, ensuring that the results meet acceptable standards across various applications.
Key parameters for method validation include:

Improving Translational Accuracy

Improving Translational Accuracy

Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...

Improving Translational Accuracy

Improving Translational Accuracy

Prediction Intervals

Prediction Intervals

The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.

Accuracy and Errors in Hypothesis Testing

Accuracy and Errors in Hypothesis Testing

Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...

Survival Tree

Survival Tree

Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Detecting Adverse Events Following Acute Care Encounters among Veterans Prescribed Non-Steroidal Anti-Inflammatory Drugs.

Journal of the American College of Emergency Physicians open·2026

Same author

Veteran Satisfaction With Lung Cancer Screening Care: A Cross-Sectional Evaluation.

Medical care·2026

Same author

Charting the future of ACMI: a report from the 2025 ACMI symposium.

Journal of the American Medical Informatics Association : JAMIA·2026

Same author

Detecting Uncoded Self-Harm in Veterans' Electronic Health Records Using Positive and Unlabeled Learning: Retrospective Cohort Study.

Journal of medical Internet research·2026

Same author

Implications of Modest Reductions in ABI Measurements for Risk of Major Adverse Limb Events Among 223,350 Veterans.

Journal of the American College of Cardiology·2026

Same author

Explainability in context: calibrating appropriate trust and reliance in artificial intelligence.

Journal of the American Medical Informatics Association : JAMIA·2026

Search research articles

Related Experiment Video

Updated: Dec 23, 2025

An R-Based Landscape Validation of a Competing Risk Model

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

Comparison of Prediction Model Performance Updating Protocols: Using a Data-Driven Testing Procedure to Guide

Sharon E Davis¹, Robert A Greevy¹, Thomas A Lasko¹

¹Vanderbilt University School of Medicine, Nashville, TN.

AMIA ... Annual Symposium Proceedings. AMIA Symposium

|April 21, 2020

Summary

A new data-driven strategy for updating clinical prediction models improves accuracy over time. This approach offers better calibration than static or annual retraining, adapting to evolving clinical environments.

More Related Videos

Design and Analysis for Fall Detection System Simplification

Design and Analysis for Fall Detection System Simplification

Published on: April 6, 2020

Related Experiment Videos

Last Updated: Dec 23, 2025

An R-Based Landscape Validation of a Competing Risk Model

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

Design and Analysis for Fall Detection System Simplification

Design and Analysis for Fall Detection System Simplification

Published on: April 6, 2020

Area of Science:

Clinical informatics
Biostatistics
Machine learning in healthcare

Background:

Prediction model accuracy declines in dynamic clinical settings.
Limited guidance exists on optimal model updating policies.
The impact of different updating strategies on model performance across various model types is underexplored.

Purpose of the Study:

To introduce and evaluate a novel data-driven updating strategy for clinical prediction models.
To compare the performance of this new strategy against baseline approaches (no updates, annual refitting).
To assess the impact of updating policies on different model types (logistic regression, L1-regularized logistic regression, random forest, neural networks).

Main Methods:

Implementation of a data-driven updating strategy using a nonparametric testing procedure.
Comparison with two baseline strategies: no model updates and annual full refitting.
Evaluation of model calibration and performance across different machine learning models.

Main Results:

The test-based updating strategy recommended intermittent recalibration.
This strategy yielded more highly calibrated predictions compared to baseline approaches.
Significant differences in updating needs (extent and timing) were observed across logistic regression, L1-regularized logistic regression, random forest, and neural network models.

Conclusions:

A data-driven model maintenance approach outperforms 'one-size-fits-all' strategies.
Intermittent recalibration, guided by data, enhances prediction model stability and accuracy over time.
Tailoring updating policies to specific model types is crucial for sustained performance in evolving clinical environments.