Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Survival Tree01:19

Survival Tree

79
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
 Building a Survival Tree
Constructing a...
79
Bias01:22

Bias

4.2K
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
4.2K
Quantifying and Rejecting Outliers: The Grubbs Test01:02

Quantifying and Rejecting Outliers: The Grubbs Test

1.5K
Sometimes, a data set can have a recorded numerical observation that greatly  deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier.  To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.5K
One-Way ANOVA: Unequal Sample Sizes01:15

One-Way ANOVA: Unequal Sample Sizes

5.7K
One-way ANOVA can be performed on three or more samples of unequal sizes. However, calculations get complicated when sample sizes are not always the same. So, while performing ANOVA with unequal samples size, the following equation is used:
5.7K
Outliers and Influential Points01:08

Outliers and Influential Points

4.0K
An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the...
4.0K
One-Way ANOVA: Equal Sample Sizes01:15

One-Way ANOVA: Equal Sample Sizes

3.3K
One-Way ANOVA can be performed on three or more samples with equal or unequal sample sizes. When one-way ANOVA is performed on two datasets with samples of equal sizes, it can be easily observed that the computed F statistic is highly sensitive to the sample mean.
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
3.3K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Reply to Du: Differences in experimental setting make validation harder.

American journal of obstetrics & gynecology MFM·2026
Same author

End-to-end evaluation of pipelines for metagenome-assembled genomes reveals hidden performance gaps.

bioRxiv : the preprint server for biology·2026
Same author

A generalizable cross-continent prediction of esophageal squamous cell carcinoma using the oral microbiome.

Communications medicine·2026
Same author

CLADES - Contrastive Learning Augmented DifferEntial Splicing with Orthologous Positive Pairs.

bioRxiv : the preprint server for biology·2026
Same author

Efficient algorithms for simulating sequences along a phylogenetic tree.

Bioinformatics (Oxford, England)·2025
Same author

Comparative metagenomics using pan-metagenomic graphs.

bioRxiv : the preprint server for biology·2025

Related Experiment Video

Updated: Jun 23, 2025

An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.0K

Distributional bias compromises leave-one-out cross-validation.

George I Austin, Itsik Pe'er, Tal Korem

    Arxiv
    |June 17, 2024
    PubMed
    Summary

    Leave-one-out cross-validation can negatively impact machine learning model evaluation due to distributional bias. A new rebalanced cross-validation method corrects this bias, improving performance assessment in data-scarce scenarios.

    Area of Science:

    • Machine Learning
    • Statistical Modeling
    • Data Science

    Background:

    • Cross-validation is standard for estimating machine learning model performance.
    • Leave-one-out cross-validation (LOOCV) is common in data-scarce settings to maximize training data.
    • LOOCV trains a model on all but one instance, testing on the held-out instance.

    Purpose of the Study:

    • To identify and characterize a performance evaluation bias in LOOCV.
    • To propose a novel cross-validation technique mitigating this bias.
    • To demonstrate the effectiveness of the proposed method across various scenarios.

    Main Methods:

    • Investigated the relationship between training fold averages and test instance labels in LOOCV.
    • Introduced a rebalanced cross-validation approach to correct for distributional bias.

    More Related Videos

    A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
    12:18

    A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment

    Published on: January 11, 2020

    7.5K
    Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
    07:35

    Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

    Published on: October 11, 2018

    7.5K

    Related Experiment Videos

    Last Updated: Jun 23, 2025

    An R-Based Landscape Validation of a Competing Risk Model
    05:37

    An R-Based Landscape Validation of a Competing Risk Model

    Published on: September 16, 2022

    2.0K
    A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
    12:18

    A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment

    Published on: January 11, 2020

    7.5K
    Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
    07:35

    Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

    Published on: October 11, 2018

    7.5K
  • Evaluated the proposed method using synthetic data, machine learning benchmarks, and real-world datasets.

    • Main Results:

    • Demonstrated that LOOCV introduces a negative correlation (distributional bias) between training and test labels.
    • Showed this bias negatively affects performance estimation and hyperparameter tuning, favoring weaker regularization.
    • The proposed rebalanced cross-validation significantly improved evaluation accuracy.

    Conclusions:

    • Distributional bias is an inherent issue in standard leave-one-out and leave-P-out cross-validation.
    • The proposed rebalanced cross-validation effectively corrects for this bias.
    • This new approach offers more reliable performance estimation and hyperparameter optimization, especially in data-limited situations.