Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Survival Tree01:19

Survival Tree

451
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
 Building a Survival Tree
Constructing a...
451
One-Way ANOVA: Unequal Sample Sizes01:15

One-Way ANOVA: Unequal Sample Sizes

6.8K
One-way ANOVA can be performed on three or more samples of unequal sizes. However, calculations get complicated when sample sizes are not always the same. So, while performing ANOVA with unequal samples size, the following equation is used:
6.8K
One-Way ANOVA: Equal Sample Sizes01:15

One-Way ANOVA: Equal Sample Sizes

4.2K
One-Way ANOVA can be performed on three or more samples with equal or unequal sample sizes. When one-way ANOVA is performed on two datasets with samples of equal sizes, it can be easily observed that the computed F statistic is highly sensitive to the sample mean.
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
4.2K
Standard Error of the Mean01:13

Standard Error of the Mean

12.7K
The sampling variability of a statistic is defined as how much the statistic varies from one sample to another. The sampling variability of a statistic is typically measured by measuring its standard error.
The standard error of the mean is an example of a standard error. It is a unique standard deviation known as the standard deviation of the sampling distribution of the mean. The standard error of the mean is a statistic that calculates how correctly a sample distribution represents a...
12.7K
Margin of Error01:27

Margin of Error

7.9K
The margin of error is also called the maximum error of an estimate. The margin of error is the maximum possible or expected difference between the observed sample parameter value and the actual population parameter value. For proportion, it is the maximum difference between the value of sample proportion obtained from the data and the true value of population proportion. As the true value of the population parameter is not known, the margin of error is calculated using the sample statistic.
7.9K
Errors In Hypothesis Tests01:14

Errors In Hypothesis Tests

6.1K
When performing a hypothesis test, there are four possible outcomes depending on the actual truth (or falseness) of the null hypothesis and the decision to reject or not.
6.1K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Innovative Clinical Trial Approach for Evaluating Digital Medical Devices Under European Fast-Track Regulatory Frameworks.

Statistics in medicine·2026
Same author

Evaluation of performance measures in predictive artificial intelligence models to support medical decisions: overview and guidance.

The Lancet. Digital health·2025
Same author

Causal Inference Methods for Combining Randomized Trials and Observational Studies: A Review.

Statistical science : a review journal of the Institute of Mathematical Statistics·2025
Same author

Advancing science- and evidence-based AI policy.

Science (New York, N.Y.)·2025
Same author

Confidence intervals for performance estimates in brain MRI segmentation.

Medical image analysis·2025
Same author

From Prediction to Prescription: Machine Learning and Causal Inference for the Heterogeneous Treatment Effect.

Annual review of biomedical data science·2025

Related Experiment Video

Updated: Feb 27, 2026

Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates
08:56

Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates

Published on: January 13, 2023

2.9K

Cross-validation failure: Small sample sizes lead to large error bars.

Gaël Varoquaux1

  • 1Parietal Project-team, INRIA Saclay-île de France, France; CEA/Neurospin bât 145, 91191 Gif-Sur-Yvette, France; Université Paris-Saclay, Saclay, France.

Neuroimage
|June 29, 2017
PubMed
Summary

Error bars in brain image analysis cross-validation are often underestimated. Small sample sizes in neuroimaging studies lead to unreliable conclusions from predictive models, necessitating larger sample sizes.

Keywords:
BiomarkersCross-validationDecodingMVPAModel selectionStatisticsfMRI

More Related Videos

An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.7K
Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model
07:13

Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model

Published on: April 18, 2025

789

Related Experiment Videos

Last Updated: Feb 27, 2026

Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates
08:56

Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates

Published on: January 13, 2023

2.9K
An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.7K
Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model
07:13

Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model

Published on: April 18, 2025

789

Area of Science:

  • Neuroimaging
  • Statistical analysis
  • Machine learning

Background:

  • Predictive models are crucial for brain image analysis, including decoding and biomarker extraction.
  • Cross-validation is the standard method for assessing the validity and utility of these models.
  • Error bars associated with cross-validation are frequently underestimated in neuroimaging research.

Purpose of the Study:

  • To highlight the issue of underestimated error bars in cross-validation for neuroimaging studies.
  • To emphasize the impact of small sample sizes on the reliability of predictive model conclusions.
  • To advocate for strategies that increase sample size in neuroimaging research.

Main Methods:

  • Analysis of error bars in cross-validation across various neuroimaging study sizes.
  • Demonstration of how standard error across folds underestimates true error.
  • Exploration of the consequences for biomarker and methods development.

Main Results:

  • Small sample sizes (e.g., 100 samples) in neuroimaging studies inherently result in large error bars (e.g., ±10%).
  • Standard error calculations across cross-validation folds significantly underestimate these error bars.
  • The reliability of conclusions drawn from predictive models is compromised.

Conclusions:

  • Underestimated error bars in cross-validation pose a significant threat to the reliability of findings in statistical brain image analysis.
  • The limited sample sizes common in neuroimaging studies exacerbate this issue, impacting biomarker discovery and methodological advancements.
  • Investigating solutions to increase sample size, while managing data heterogeneity, is critical for robust neuroimaging research.