Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Regression Toward the Mean01:52

Regression Toward the Mean

6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
Kaplan-Meier Approach01:24

Kaplan-Meier Approach

190
The Kaplan-Meier estimator is a non-parametric method used to estimate the survival function from time-to-event data. In medical research, it is frequently employed to measure the proportion of patients surviving for a certain period after treatment. This estimator is fundamental in analyzing time-to-event data, making it indispensable in clinical trials, epidemiological studies, and reliability engineering. By estimating survival probabilities, researchers can evaluate treatment effectiveness,...
190
Statistical Methods to Analyze Parametric Data: Student t-Test and Goodness-of-Fit Test01:09

Statistical Methods to Analyze Parametric Data: Student t-Test and Goodness-of-Fit Test

1.7K
In parametric statistics, two fundamental tests stand out for their utility and wide application: the Student's t-test and goodness-of-fit tests. These tests provide researchers with a robust method for drawing insights from data, testing hypotheses, and making informed decisions based on their findings.
The Student's t-test is a statistical test that examines if there is a statistically significant difference between the means of two groups. This test is instrumental when dealing with...
1.7K
Comparing the Survival Analysis of Two or More Groups01:20

Comparing the Survival Analysis of Two or More Groups

227
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and...
227
Pharmacokinetic Models: Comparison and Selection Criterion01:26

Pharmacokinetic Models: Comparison and Selection Criterion

109
Physiological and compartmental models are valuable tools used in studying biological systems. These models rely on differential equations to maintain mass balance within the system, ensuring an accurate representation of the dynamic processes at play.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
109
Mechanistic Models: Compartment Models in Individual and Population Analysis01:23

Mechanistic Models: Compartment Models in Individual and Population Analysis

65
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
65

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Muscle morphology and function following a transfemoral amputation: A systematic review.

Canadian prosthetics & orthotics journal·2026
Same author

Corrigendum: Relative aerobic load of daily activities in individuals with lower limb amputation: A pilot study.

Canadian prosthetics & orthotics journal·2026
Same author

Development and validation of a clinical prediction model for hospitalization after emergency department admission in patients with cancer.

ESMO real world data and digital oncology·2026
Same author

Relative Aerobic Load of Daily Activities in Individuals with Lower Limb Amputation: A pilot study.

Canadian prosthetics & orthotics journal·2026
Same author

Widely tunable and narrow-linewidth violet lasers enabled by UV-transparent materials.

Nature communications·2025
Same author

Stepwise development of an implementation protocol to support the prescription of Exercise = Medicine by clinicians using the Implementation Mapping approach.

Frontiers in health services·2025
Same journal

Leveraging generative artificial intelligence for the development of non-interventional research study protocols: a proof-of-concept feasibility study.

BMC medical research methodology·2026
Same journal

Variable selection for clinical prediction models in low-dimensional data - a simulation study comparing traditional regression and machine learning methods.

BMC medical research methodology·2026
Same journal

Integrating health economics and implementation science: a call to action.

BMC medical research methodology·2026
Same journal

Methods for incorporating test result information within the high-dimensional propensity score framework: application in UK electronic health record data.

BMC medical research methodology·2026
Same journal

Sparse multi-way DMDC for longitudinal classification in high dimension low sample size data.

BMC medical research methodology·2026
Same journal

Tree-based exploratory identification of predictive biomarkers in non-randomized data.

BMC medical research methodology·2026
See all related articles

Related Experiment Video

Updated: Jul 24, 2025

Predicting Treatment Response to Image-Guided Therapies Using Machine Learning: An Example for Trans-Arterial Treatment of Hepatocellular Carcinoma
04:09

Predicting Treatment Response to Image-Guided Therapies Using Machine Learning: An Example for Trans-Arterial Treatment of Hepatocellular Carcinoma

Published on: October 10, 2018

8.3K

Performance metrics for models designed to predict treatment effect.

C C H M Maas1, D M Kent2, M C Hughes2

  • 1Department of Public Health, Erasmus University Medical Center, Doctor Molewaterplein 40, 3015 GD, Rotterdam, Netherlands. c.h.m.maas@erasmusmc.nl.

BMC Medical Research Methodology
|July 8, 2023
PubMed
Summary
This summary is machine-generated.

New metrics evaluate individualized treatment effect models in randomized clinical trials (RCTs). These metrics assess calibration and overall performance, addressing limitations of existing methods for predicting treatment effects.

Keywords:
Causal forestHeterogeneous treatment effectLogistic regressionPrediction models

More Related Videos

Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index
06:55

Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index

Published on: January 8, 2020

14.5K
An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.1K

Related Experiment Videos

Last Updated: Jul 24, 2025

Predicting Treatment Response to Image-Guided Therapies Using Machine Learning: An Example for Trans-Arterial Treatment of Hepatocellular Carcinoma
04:09

Predicting Treatment Response to Image-Guided Therapies Using Machine Learning: An Example for Trans-Arterial Treatment of Hepatocellular Carcinoma

Published on: October 10, 2018

8.3K
Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index
06:55

Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index

Published on: January 8, 2020

14.5K
An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.1K

Area of Science:

  • Biostatistics
  • Clinical Trials
  • Machine Learning

Background:

  • Predicting individualized treatment effects is complex due to unobservable counterfactual outcomes.
  • Existing C-for-benefit metric measures discriminative ability but lacks calibration and overall performance measures.
  • There is a need for robust metrics to evaluate treatment effect prediction models in clinical research.

Purpose of the Study:

  • To propose novel metrics for assessing the calibration and overall performance of models predicting individualized treatment effects.
  • To extend the evaluation of treatment effect models beyond discriminative ability.
  • To provide tools for more accurate model assessment in randomized clinical trials (RCTs).

Main Methods:

  • Defined observed pairwise treatment effect using matched untreated and treated patients based on Mahalanobis distance.
  • Introduced E-for-benefit metrics (E_avg, E_50, E_90) quantifying prediction accuracy against smoothed observed effects.
  • Developed cross-entropy-for-benefit and Brier-for-benefit metrics for assessing prediction error.
  • Validated metrics via simulation comparing "optimal" and "perturbed" models.
  • Applied metrics to the Diabetes Prevention Program data using diverse modeling approaches (risk modeling, effect modeling, causal forest).

Main Results:

  • "Perturbed models" consistently showed worse performance metrics than the "optimal model" across all proposed metrics.
  • Simulation results demonstrated the sensitivity of the new metrics to model performance differences.
  • Case study showed similar calibration, discriminative ability, and overall performance for different modeling approaches.
  • The proposed metrics were implemented in the R-package "HTEPredictionMetrics".

Conclusions:

  • The newly proposed metrics effectively assess calibration and overall performance of treatment effect prediction models in RCTs.
  • These metrics offer a valuable addition to the toolkit for evaluating clinical prediction models.
  • The availability of the R-package facilitates the practical application of these performance metrics.