Evaluating two small-sample corrections for fixed-effects standard errors and inferences in multilevel models with...

Area of Science:

• Psychological research methodology
• Statistical modeling
• Quantitative psychology

Background:

• Multilevel modeling (MLM) is standard for clustered data in psychology.
• Homogeneity of variance is a key MLM assumption, often violated in practice.
• Violations lead to misestimated standard errors, impacting inference validity.

Purpose of the Study:

• Compare Kenward-Roger (KR) and adjusted cluster-robust standard errors (CR-SEs) for heteroscedastic clustered data.
• Evaluate performance across ordinary least squares (OLS), random intercept (RI), and random slope (RS) models.
• Identify optimal methods for accurate statistical inference in small, non-normal sample sizes.

Main Methods:

• Monte Carlo simulation study.
• Analysis of small, heteroscedastic, clustered data.
• Comparison of KR adjustment with RS models versus adjusted CR-SEs with OLS, RI, and RS models.

Main Results:

• KR with RS models showed significant bias and inflated Type I errors for between-cluster effects under heteroscedasticity.
• Adjusted CR-SEs demonstrated acceptable bias and controlled Type I error rates across all models.
• Adjusted CR-SEs with RS models provided higher power for between-cluster effects.

Conclusions:

• Adjusted CR-SEs are recommended for accurate multilevel modeling inferences, especially with heteroscedasticity.
• For within-cluster effects, any model with adjusted CR-SEs is suitable.
• For between-cluster effects, adjusted CR-SEs with RS models enhance power and mitigate heterogeneity issues.