Fixed Effects or Mixed Effects Classifiers? Evidence From Simulated and Archival Data

Area of Science:

• Statistical modeling
• Machine learning
• Educational research

Background:

• Multilevel data structures are common in social sciences and education.
• Predictive classification models are crucial for understanding complex phenomena.
• Choosing between fixed and mixed effects models can impact classification accuracy.

Purpose of the Study:

• To compare the predictive classification performance of fixed effects models versus mixed effects models.
• To evaluate logistic regression and random forests within both fixed and mixed effects frameworks.
• To assess model performance using both simulated data and real-world educational data.

Main Methods:

• Monte Carlo simulation to generate and analyze multilevel data.
• Comparison of fixed effects logistic regression, mixed effects logistic regression, and random forests.
• Application to U.S. PISA data for predicting student retention.

Main Results:

• Fixed effects models demonstrated comparable performance to mixed effects models.
• Model performance was consistent across simulation studies and the PISA dataset analysis.
• The type of predictors and the data structure were more influential than the model type itself.

Conclusions:

• Fixed effects models are a viable alternative to mixed effects models for predictive classification with multilevel data.
• Researchers should prioritize careful consideration of predictor variables and data structure.
• Model selection should be guided by data characteristics rather than a default preference for mixed effects models.