Assessing Model Selection Uncertainty Using a Bootstrap Approach: An update.

Area of Science:

• Behavioral Sciences
• Psychometrics
• Statistical Modeling

Background:

• Model comparison in behavioral sciences typically relies on fit indices (e.g., AIC, BIC) to select the best population model.
• Current methods often overlook sampling variability, which can lead to different model selections across samples.
• A prior study introduced a bootstrap approach to quantify model selection uncertainty.

Purpose of the Study:

• To evaluate the effectiveness of a bootstrap approach for assessing model selection uncertainty.
• To examine its utility in multi-group and mixture model comparisons.
• To determine if bootstrap selection rates offer additional information beyond standard fit index differences.

Main Methods:

• Conducted a series of simulation studies.
• Applied the bootstrap approach to multi-group model comparisons.
• Applied the bootstrap approach to mixture model comparisons.

Main Results:

• Bootstrap selection rates provide valuable information for model selection.
• This information is supplementary to differences in Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC).
• The bootstrap approach is useful in both multi-group and mixture model contexts.

Conclusions:

• The proposed bootstrap method enhances the understanding of model selection uncertainty in statistical modeling.
• It offers a more robust approach compared to relying solely on AIC and BIC differences.
• This technique is particularly beneficial for complex models in behavioral research.