Model Selection and Model Averaging for Mixed-Effects Models with Crossed Random Effects for Subjects and Items

Area of Science:

• Statistics
• Psychometrics
• Experimental Design

Background:

• Experimental research frequently involves random effects from subjects and items.
• Mixed-effects models (MEMs) with crossed random effects are standard for analyzing such variability.
• Incorrect parameterization of MEMs' random structure can bias standard error (SE) estimations for fixed effects.

Purpose of the Study:

• To investigate the impact of model selection and model averaging on standard error bias in mixed-effects models.
• To examine how sample size and random slope variance influence model selection accuracy and SE bias.
• To compare the performance of Maximum Likelihood (ML) and Restricted Maximum Likelihood (REML) estimators regarding SE bias.

Main Methods:

• A simulation study was conducted to evaluate model selection (likelihood-ratio tests, AIC, BIC) and model averaging (Akaike weights).
• The study analyzed the influence of sample size and variance of random slopes on model selection and SE bias.
• Comparisons were made between ML and REML estimators, focusing on SE bias for fixed effects.

Main Results:

• Model selection accuracy was consistent across different strategies and estimators (ML, REML).
• Sample size and random slope variance significantly impacted model selection and SE bias.
• Underestimation of SEs was observed for within-subjects effects with small item counts and large item random slopes; this bias was greater with ML than REML.

Conclusions:

• Neither model selection nor model averaging consistently eliminated standard error bias in MEMs.
• Sample size and random slope variance are critical factors influencing SE bias, interacting with the choice of estimator.
• The high variability in SE bias, particularly with smaller sample sizes or larger random slopes, can lead to unacceptable bias rates in replications.