Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Comparing the Survival Analysis of Two or More Groups01:20

Comparing the Survival Analysis of Two or More Groups

253
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and...
253
Study Design in Statistics01:15

Study Design in Statistics

8.4K
A study design is a set of techniques that allow a researcher to collect and analyze data from different variables defined for a specific research problem. Statistics is commonly for effective study design and more robust experiments,
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer more effective at growing roses than another? Is fatigue as dangerous to a driver as the influence of alcohol? Questions like these are answered using randomized experiments with proper...
8.4K
One-Way ANOVA: Unequal Sample Sizes01:15

One-Way ANOVA: Unequal Sample Sizes

5.9K
One-way ANOVA can be performed on three or more samples of unequal sizes. However, calculations get complicated when sample sizes are not always the same. So, while performing ANOVA with unequal samples size, the following equation is used:
5.9K
One-Way ANOVA: Equal Sample Sizes01:15

One-Way ANOVA: Equal Sample Sizes

3.4K
One-Way ANOVA can be performed on three or more samples with equal or unequal sample sizes. When one-way ANOVA is performed on two datasets with samples of equal sizes, it can be easily observed that the computed F statistic is highly sensitive to the sample mean.
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
3.4K
Censoring Survival Data01:09

Censoring Survival Data

198
Survival analysis is a statistical method used to analyze time-to-event data, often employed in fields such as medicine, engineering, and social sciences. One of the key challenges in survival analysis is dealing with incomplete data, a phenomenon known as "censoring." Censoring occurs when the event of interest (such as death, relapse, or system failure) has not occurred for some individuals by the end of the study period or is otherwise unobservable, and it might have many different...
198
Truncation in Survival Analysis01:09

Truncation in Survival Analysis

275
Truncation in survival analysis refers to the exclusion of individuals or events from the dataset based on specific criteria related to the time of the event. This exclusion can happen in two primary forms: left truncation and right truncation.
Left truncation occurs when individuals who experienced the event of interest before a certain time are not included in the study. This is often due to a "delayed entry" into the study where only those who survive until a certain entry point are...
275

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Self-Reported Penicillin Allergy, Alternative Antibiotics, and Implant Outcomes: A Retrospective Study.

Clinical implant dentistry and related research·2026
Same author

Statistical inference for saved time based on disease progression curves in Alzheimer's disease research.

Contemporary clinical trials communications·2026
Same author

Statistical models for Alzheimer's disease clinical trials: Lessons learned from the DIAN-TU Platform Trial.

Journal of Alzheimer's disease : JAD·2026
Same author

Penalized estimation of linear transformation models for interval-censored data with time-dependent covariates.

Statistical methods in medical research·2026
Same author

Impact of male genital tract infections on semen quality: a systematic review and meta-analysis.

Fertility and sterility·2026
Same author

Testing disease progression under the proportional reduction in decline in Alzheimer's disease studies.

Journal of applied statistics·2026

Related Experiment Video

Updated: Aug 25, 2025

An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.2K

Monte Carlo cross-validation for a study with binary outcome and limited sample size.

Guogen Shan1

  • 1Department of Biostatistics, University of Florida, Gainesville, FL, 32610, USA. gshan@ufl.edu.

BMC Medical Informatics and Decision Making
|October 17, 2022
PubMed
Summary

Monte Carlo cross-validation (MCCV) offers slightly higher accuracy than standard cross-validation (CV) for limited sample sizes. Increasing MCCV simulations improves reliability, making it a robust choice for machine learning model evaluation.

Keywords:
Alzheimer’s diseaseBinary outcomeCross-validationMachine learningMonte Carlo cross-validation

More Related Videos

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

7.6K
A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
12:18

A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment

Published on: January 11, 2020

7.6K

Related Experiment Videos

Last Updated: Aug 25, 2025

An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.2K
Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

7.6K
A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
12:18

A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment

Published on: January 11, 2020

7.6K

Area of Science:

  • Machine Learning
  • Biostatistics
  • Computational Science

Background:

  • Cross-validation (CV) is essential for evaluating machine learning models with limited data.
  • Leave-one-out CV can result in a very small number of validation rounds.
  • Monte Carlo cross-validation (MCCV) offers flexibility in the number of simulations.

Purpose of the Study:

  • To compare the accuracy of MCCV and CV for binary outcomes.
  • To assess the impact of simulation numbers on MCCV reliability.
  • To illustrate the practical differences between MCCV and CV using real-world examples.

Main Methods:

  • Extensive simulation studies were conducted.
  • Comparison focused on models with binary outcomes.
  • Both MCCV and CV were evaluated with an equal number of simulations.

Main Results:

  • MCCV demonstrated generally higher accuracy than CV, though the difference was marginal.
  • Performance was similar between MCCV and CV when dealing with large sample sizes.
  • Increasing the number of MCCV simulations led to more reliable performance metrics.

Conclusions:

  • MCCV provides a reliable alternative to CV, especially for studies with limited sample sizes.
  • The accuracy gains of MCCV over CV are modest but consistent.
  • MCCV's reliability increases with a higher number of simulation iterations.