Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Comparing the Survival Analysis of Two or More Groups01:20

Comparing the Survival Analysis of Two or More Groups

195
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and...
195
Statistical Inference Techniques in Hypothesis Testing: Parametric Versus Nonparametric Data01:16

Statistical Inference Techniques in Hypothesis Testing: Parametric Versus Nonparametric Data

130
Statistical inference techniques, paramount in hypothesis testing, differentiate into two broad categories: parametric and nonparametric statistics.
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance,...
130
Hazard Ratio01:12

Hazard Ratio

128
The hazard ratio (HR) is a widely used measure in clinical trials to compare the risk of events, such as death or disease recurrence, between two groups over time. It reflects the ratio of hazard rates—the instantaneous risk of the event occurring—between a treatment group and a control group. This measure provides valuable insights into the relative effectiveness of a treatment by assessing how the risk of an event differs between the two groups.
For example, in a clinical trial...
128
Receiver Operating Characteristic Plot01:15

Receiver Operating Characteristic Plot

183
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
183
Statistical Methods to Analyze Parametric Data: Student t-Test and Goodness-of-Fit Test01:09

Statistical Methods to Analyze Parametric Data: Student t-Test and Goodness-of-Fit Test

1.6K
In parametric statistics, two fundamental tests stand out for their utility and wide application: the Student's t-test and goodness-of-fit tests. These tests provide researchers with a robust method for drawing insights from data, testing hypotheses, and making informed decisions based on their findings.
The Student's t-test is a statistical test that examines if there is a statistically significant difference between the means of two groups. This test is instrumental when dealing with...
1.6K
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

56
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
56

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

When to Adjust for Multiple Testing: A Unifying Guiding Principle.

Biometrical journal. Biometrische Zeitschrift·2026
Same author

Terms, definitions and measurements to describe the sonographic features of adnexal tumors: updated consensus opinion from the International Ovarian Tumor Analysis (IOTA) Group.

Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology·2026
Same author

Developing and validating ultrasound-based machine-learning models incorporating radiomics features to predict malignancy in adnexal masses.

Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology·2026
Same author

Signal or noise? Evaluating commonly used attribution methods for explaining deep neural networks in electrocardiogram classification.

European heart journal. Digital health·2026
Same author

Diagnostic tests for ovarian cancer in premenopausal women with non-specific symptoms (ROCkeTS): prospective, multicentre, cohort study.

BMJ (Clinical research ed.)·2026
Same author

ADNEX-AI: automated extraction of ultrasound predictors for interpretable ovarian cancer risk stratification.

NPJ precision oncology·2025

Related Experiment Video

Updated: Jul 6, 2025

Machine Learning Algorithms for Early Detection of Bone Metastases in an Experimental Rat Model
07:15

Machine Learning Algorithms for Early Detection of Bone Metastases in an Experimental Rat Model

Published on: August 16, 2020

6.8K

A comparison of hyperparameter tuning procedures for clinical prediction models: A simulation study.

Zoë S Dunias1, Ben Van Calster2,3, Dirk Timmerman2,4

  • 1Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands.

Statistics in Medicine
|January 8, 2024
PubMed
Summary

Standard cross-validation (5-fold or 10-fold) effectively tunes clinical prediction models, outperforming other methods. Avoid the one-standard-error rule (1SE CV) for cross-validation, as it can harm model calibration.

Keywords:
Random Forestcross-validationhyperparameter tuningpenalized regressionprediction models

More Related Videos

An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.1K
Predicting Treatment Response to Image-Guided Therapies Using Machine Learning: An Example for Trans-Arterial Treatment of Hepatocellular Carcinoma
04:09

Predicting Treatment Response to Image-Guided Therapies Using Machine Learning: An Example for Trans-Arterial Treatment of Hepatocellular Carcinoma

Published on: October 10, 2018

8.3K

Related Experiment Videos

Last Updated: Jul 6, 2025

Machine Learning Algorithms for Early Detection of Bone Metastases in an Experimental Rat Model
07:15

Machine Learning Algorithms for Early Detection of Bone Metastases in an Experimental Rat Model

Published on: August 16, 2020

6.8K
An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.1K
Predicting Treatment Response to Image-Guided Therapies Using Machine Learning: An Example for Trans-Arterial Treatment of Hepatocellular Carcinoma
04:09

Predicting Treatment Response to Image-Guided Therapies Using Machine Learning: An Example for Trans-Arterial Treatment of Hepatocellular Carcinoma

Published on: October 10, 2018

8.3K

Area of Science:

  • Biostatistics
  • Machine Learning in Healthcare
  • Clinical Prediction Modeling

Background:

  • Hyperparameter tuning is crucial for optimizing clinical risk prediction models.
  • Methods like Ridge, Lasso, Elastic Net, and Random Forest are commonly used.
  • Evaluating tuning procedures is essential for reliable out-of-sample performance.

Purpose of the Study:

  • To systematically compare hyperparameter tuning procedures for clinical prediction models.
  • To evaluate the impact of sample size, predictor count, and events fraction on tuning performance.
  • To assess discrimination, calibration, and prediction error of various tuning methods.

Main Methods:

  • Extensive simulations were conducted on low-dimensional data.
  • Compared Ridge, Lasso, Elastic Net, and Random Forest models.
  • Evaluated standard cross-validation (5-fold, 10-fold, repeated), bootstrap, and the one-standard-error rule (1SE CV).

Main Results:

  • Standard cross-validation (5-fold and 10-fold) demonstrated superior calibration and overall performance.
  • The 1SE CV rule frequently led to significant miscalibration.
  • Bootstrap tuning showed a tendency towards more severe miscalibration than standard CV.
  • Differences in performance were more pronounced with smaller sample sizes and lower events fractions.

Conclusions:

  • Standard 5-fold or 10-fold cross-validation is recommended for tuning clinical prediction models to minimize prediction error.
  • The 1SE CV rule should be used with caution in low-dimensional settings due to potential calibration issues.
  • Choice of tuning procedure significantly impacts model predictive performance, especially in challenging data scenarios.