Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Statistical Analysis: Overview01:11

Statistical Analysis: Overview

8.2K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
8.2K
Accuracy and Errors in Hypothesis Testing01:13

Accuracy and Errors in Hypothesis Testing

347
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
347
Uncertainty in Measurement: Accuracy and Precision03:37

Uncertainty in Measurement: Accuracy and Precision

87.7K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value. 
87.7K
Mechanistic Models: Compartment Models in Individual and Population Analysis01:23

Mechanistic Models: Compartment Models in Individual and Population Analysis

89
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
89
Variation01:19

Variation

7.2K
An important characteristic of any set of data is the variation in the data. In some data sets, the data values are concentrated closely near the mean; in other data sets, the data values are more widely spread out from the mean. The most common measure of variation, or spread, is the standard deviation, which is the square root of variance.
When independent and dependent variables are plotted on a scatter plot, the slope of a line is a value that describes the rate of change between the two...
7.2K
Detection of Gross Error: The Q Test01:00

Detection of Gross Error: The Q Test

6.4K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.4K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

The Risks of Risk Assessment: Causal Blind Spots When Using Prediction Models for Treatment Decisions.

Annals of internal medicine·2025
Same author

Replication studies in the Netherlands: Lessons learned and recommendations for funders, publishers and editors, and universities.

Accountability in research·2024
Same author

Safety of treating acute pulmonary embolism at home: an individual patient data meta-analysis.

European heart journal·2024
Same author

Replicability of simulation studies for the investigation of statistical methods: the RepliSims project.

Royal Society open science·2024
Same author

How to assess applicability and methodological quality of comparative studies of operative interventions in orthopedic trauma surgery.

European journal of trauma and emergency surgery : official publication of the European Trauma Society·2022
Same author

A comparison of full model specification and backward elimination of potential confounders when estimating marginal and conditional causal effects on binary outcomes from observational data.

Biometrical journal. Biometrische Zeitschrift·2022

Related Experiment Video

Updated: Sep 27, 2025

An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.2K

Quantitative prediction error analysis to investigate predictive performance under predictor measurement

Kim Luijken1, Jia Song2, Rolf H H Groenwold2,3

  • 1Department of Clinical Epidemiology, Leiden University Medical Center, Leiden, the Netherlands. k.luijken@lumc.nl.

Diagnostic and Prognostic Research
|April 7, 2022
PubMed
Summary

Predictor measurement heterogeneity in prognostic models significantly reduces predictive performance. Quantifying this impact is crucial for reliable model implementation in clinical settings.

Keywords:
CalibrationExternal validationMeasurement heterogeneityPrognostic model

More Related Videos

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach
04:35

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach

Published on: July 3, 2020

3.4K
Probing the Limits of Egg Recognition Using Egg Rejection Experiments Along Phenotypic Gradients
07:34

Probing the Limits of Egg Recognition Using Egg Rejection Experiments Along Phenotypic Gradients

Published on: August 22, 2018

8.3K

Related Experiment Videos

Last Updated: Sep 27, 2025

An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.2K
Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach
04:35

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach

Published on: July 3, 2020

3.4K
Probing the Limits of Egg Recognition Using Egg Rejection Experiments Along Phenotypic Gradients
07:34

Probing the Limits of Egg Recognition Using Egg Rejection Experiments Along Phenotypic Gradients

Published on: August 22, 2018

8.3K

Area of Science:

  • Biostatistics
  • Epidemiology
  • Health Informatics

Background:

  • Prognostic prediction models are vital, but their performance can degrade when predictor variables are measured differently in validation versus real-world implementation settings (predictor measurement heterogeneity).
  • Accurate inference of model performance at implementation is necessary when such heterogeneity is anticipated.

Purpose of the Study:

  • To propose and illustrate an analysis for quantifying the impact of anticipated predictor measurement heterogeneity on prognostic model performance.
  • To assess how predictor measurement heterogeneity affects model performance in time-to-event outcome data.

Main Methods:

  • A simulation study was performed to evaluate the effects of predictor measurement heterogeneity across validation and implementation settings.
  • Quantitative prediction error analysis was employed and demonstrated using a case study predicting type 2 diabetes risk, focusing on body mass index measurement heterogeneity.

Main Results:

  • Simulation results indicated poor calibration-in-the-large and reduced overall accuracy across all scenarios of predictor measurement heterogeneity.
  • Model discrimination was observed to decrease as random predictor measurement heterogeneity increased.

Conclusions:

  • Heterogeneity in predictor measurements between validation and implementation settings diminishes the predictive performance of prognostic models for time-to-event outcomes.
  • Consideration of the target clinical setting during model validation is essential, and analyses should quantify the impact of anticipated heterogeneity on implementation performance.