Related Experiment Video
Updated: Feb 27, 2026
05:37
An R-Based Landscape Validation of a Competing Risk Model
Published on: September 16, 2022
2.7K
Inference Based on the Best-Fitting Model can Contribute to the Replication Crisis: Assessing Model Selection
Gitta H Lubke1,2, Ian Campbell1
1University of Notre Dame.
Structural Equation Modeling : a Multidisciplinary Journal
|July 1, 2017
Summary
Ignoring model selection uncertainty inflates Type I errors. A bootstrap approach quantifies this uncertainty, estimating model replication probability and improving statistical inference in data analysis.
Area of Science:
- Statistics
- Psychometrics
- Data Analysis
Background:
- Statistical inference often relies on a single best-fitting model.
- Using the same data for model selection and inference overlooks model selection uncertainty, inflating Type I errors.
Purpose of the Study:
- To illustrate the impact of ignoring model selection uncertainty on statistical inference.
- To propose and demonstrate a bootstrap-based method for quantifying model selection uncertainty.
Main Methods:
- Illustrating Type I error inflation using simulation or examples.
- Developing a bootstrap approach to estimate model selection rates.
- Applying the bootstrap method to growth mixture modeling and measurement invariance analysis.
Main Results:
- Demonstrated inflation of Type I errors when model selection and inference use the same data.
- Proposed bootstrap method effectively quantifies model selection uncertainty.
- Selection rates from the bootstrap method can estimate model replication probability.
Conclusions:
- Quantifying model selection uncertainty is crucial for accurate statistical inference.
- Bootstrap-based methods offer a practical solution for addressing model selection uncertainty.
- The proposed approach enhances the reliability of findings in complex statistical analyses.
More Related Videos
04:35Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach
Published on: July 3, 2020
3.8K
20:24Characterization of Complex Systems Using the Design of Experiments Approach: Transient Protein Expression in Tobacco as a Case Study
Published on: January 31, 2014
17.2K
Related Concept Videos
01:24Bootstrapping
857
The term "bootstrap" originated in the 19th century as a metaphor for self-improvement or achieving something independently, without external assistance. This concept extends to statistical bootstrapping, a self-contained method for estimating population parameters through resampling, even though it can be computationally intensive. Developed by the American statistician Dr. Bradley Efron in 1979, bootstrapping provides a robust way to perform inference when the original sample size is...
857
01:19Survival Tree
451
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
451
01:26Pharmacokinetic Models: Comparison and Selection Criterion
426
Physiological and compartmental models are valuable tools used in studying biological systems. These models rely on differential equations to maintain mass balance within the system, ensuring an accurate representation of the dynamic processes at play.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
426
01:23Mechanistic Models: Compartment Models in Individual and Population Analysis
308
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
308
01:13Accuracy and Errors in Hypothesis Testing
636
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
636
01:15Assumptions of Survival Analysis
466
Survival models analyze the time until one or more events occur, such as death in biological organisms or failure in mechanical systems. These models are widely used across fields like medicine, biology, engineering, and public health to study time-to-event phenomena. To ensure accurate results, survival analysis relies on key assumptions and careful study design.
466