Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Bootstrapping01:24

Bootstrapping

667
The term "bootstrap" originated in the 19th century as a metaphor for self-improvement or achieving something independently, without external assistance. This concept extends to statistical bootstrapping, a self-contained method for estimating population parameters through resampling, even though it can be computationally intensive. Developed by the American statistician Dr. Bradley Efron in 1979, bootstrapping provides a robust way to perform inference when the original sample size is...
667
Accuracy and Errors in Hypothesis Testing01:13

Accuracy and Errors in Hypothesis Testing

280
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
280
Errors In Hypothesis Tests01:14

Errors In Hypothesis Tests

4.5K
When performing a hypothesis test, there are four possible outcomes depending on the actual truth (or falseness) of the null hypothesis and the decision to reject or not.
4.5K
Quantifying and Rejecting Outliers: The Grubbs Test01:02

Quantifying and Rejecting Outliers: The Grubbs Test

1.9K
Sometimes, a data set can have a recorded numerical observation that greatly  deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier.  To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.9K
Random and Systematic Errors01:20

Random and Systematic Errors

12.3K
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...
12.3K
Contaminants and Errors01:16

Contaminants and Errors

137
Effective sample preparation is crucial for accurate and reliable laboratory analysis. During this process, two significant sources of error can arise: concentration bias from improper sample splitting and contamination caused by methods used to reduce particle size, such as grinding or homogenization. Identifying and minimizing these potential errors is crucial to ensuring the validity of the analysis.
Another key consideration is determining the appropriate number of samples required to...
137

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

A scalable exemplar-based method for aligning biological taxonomies.

Biodiversity data journal·2026
Same author

Indigenous peoples and local communities as partners in the sequencing of global eukaryotic biodiversity.

npj biodiversity·2025
Same author

A systematic review of the distribution and prevalence of viruses detected in the Peromyscus maniculatus species complex (Rodentia: Cricetidae).

PLoS pathogens·2025
Same author

Living Fossil: A Metaphor's Travels Across Popular Culture and the Foundations of Darwinian Evolution and Anthropology.

Journal of the history of biology·2025
Same author

A Systematic Review of the Distribution and Prevalence of Viruses Detected in the <i>Peromyscus maniculatus</i> Species Complex (Rodentia: Cricetidae).

bioRxiv : the preprint server for biology·2024
Same author

Indigenous peoples and local communities as partners in the sequencing of global eukaryotic biodiversity.

npj biodiversity·2024

Related Experiment Video

Updated: Sep 8, 2025

An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.2K

Controlling the error probabilities of model selection information criteria using bootstrapping.

Michael Cullan1, Scott Lidgard2, Beckett Sterner3

  • 1School of Mathematics and Statistical Sciences, Arizona State University, Phoenix, AZ, USA.

Journal of Applied Statistics
|June 16, 2022
PubMed
Summary

The Akaike Information Criterion (AIC) method offers model comparison without a null model. The new Error Control for Information Criteria (ECIC) method provides Type-I error control for AIC, enhancing model selection reliability.

Keywords:
Error statisticsNeyman–Pearson classificationbootstraphypothesis testingnon-nested models

More Related Videos

Establishing a Competing Risk Regression Nomogram Model for Survival Data
04:57

Establishing a Competing Risk Regression Nomogram Model for Survival Data

Published on: October 23, 2020

10.3K
Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach
04:35

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach

Published on: July 3, 2020

3.4K

Related Experiment Videos

Last Updated: Sep 8, 2025

An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.2K
Establishing a Competing Risk Regression Nomogram Model for Survival Data
04:57

Establishing a Competing Risk Regression Nomogram Model for Survival Data

Published on: October 23, 2020

10.3K
Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach
04:35

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach

Published on: July 3, 2020

3.4K

Area of Science:

  • Statistics
  • Computational Statistics
  • Econometrics

Background:

  • Information criteria like AIC are widely used for comparing non-nested statistical models.
  • Current information-theoretic model selection lacks explicit error rate control, unlike classical hypothesis testing.
  • Extending error control concepts to multi-model selection without a null hypothesis is a significant challenge.

Purpose of the Study:

  • To extend the concepts of Type-I and Type-II errors to scenarios involving more than two models.
  • To introduce the Error Control for Information Criteria (ECIC) method for robust model selection.
  • To provide a bootstrap-based approach for controlling Type-I error rates in information-theoretic model selection.

Main Methods:

  • Extension of Type-I and Type-II error definitions to multi-model comparisons.
  • Development of the Error Control for Information Criteria (ECIC) method.
  • Utilizing bootstrap resampling and Difference of Goodness of Fit (DGOF) distributions for error control.

Main Results:

  • Demonstrated the feasibility of extending error control to information criteria.
  • ECIC method successfully controls Type-I error rates in model selection.
  • The method shows value in time series and regression analyses with simulated and empirical data.

Conclusions:

  • The ECIC method offers a novel approach to enhance the reliability of model selection using information criteria.
  • ECIC provides explicit control over Type-I error rates, a crucial feature for statistical inference.
  • The publicly available R package facilitates the application of ECIC in diverse research areas.