Bayesian Prior Choice in IRT Estimation Using MCMC and Variational Bayes.

Area of Science:

• Psychometrics
• Statistical Modeling
• Machine Learning

Background:

• Bayesian estimation is crucial for parameter recovery in statistical models.
• Prior distributions significantly influence Bayesian estimation outcomes.
• Comparing Markov chain Monte Carlo (MCMC) and Variational Bayesian (VB) methods is essential for understanding their performance.

Purpose of the Study:

• To investigate the impact of different prior distributions (matched, vague, hierarchical) on Bayesian parameter recovery.
• To compare the performance of Markov chain Monte Carlo (MCMC) and Variational Bayesian (VB) estimation methods.
• To evaluate estimation accuracy using Conditional Maximum Likelihood (CML) and Marginal Maximum Likelihood (MML) as baselines.

Main Methods:

• Employed Bayesian estimation using Markov chain Monte Monte Carlo (MCMC) and Variational Bayesian (VB) approaches.
• Investigated three prior distributions: matched, standard vague, and hierarchical.
• Utilized Conditional Maximum Likelihood (CML) and Marginal Maximum Likelihood (MML) for comparative analysis.

Main Results:

• Vague priors led to significant errors and convergence issues, deeming them unsuitable.
• Hierarchical and matched priors demonstrated the lowest root mean squared errors (RMSEs) for ability estimates in both MCMC and VB.
• Variational Bayesian (VB) estimation generally yielded lower standard errors (SEs) compared to MCMC, with VB-hierarchical and VB-matched performing optimally.

Conclusions:

• Variational Bayesian (VB) estimation with hierarchical priors is recommended for its superior accuracy, cost-effectiveness, and time efficiency.
• Matched priors also performed well with both MCMC and VB methods.
• Standard vague priors should be avoided due to poor performance and convergence problems.