Multidimensional Bayesian adaptive testing

Area of Science:

• Psychometrics
• Statistical modeling
• Educational measurement

Background:

• Conventional multidimensional adaptive testing (MAT) relies on point estimates for item and person parameters.
• This approach may not fully capture the uncertainty inherent in parameter estimation.
• Limitations exist in current MAT methods, particularly concerning the accuracy of final person parameter estimates.

Purpose of the Study:

• To introduce and evaluate a fully Bayesian approach to multidimensional adaptive testing (MBAT).
• To compare the performance of MBAT against conventional MAT using Monte Carlo simulations.
• To assess the impact of calibration sample size, test length, and trait level on MBAT and MAT performance.

Main Methods:

• A Monte Carlo simulation study was designed with a four-factorial structure.
• Factors included calibration sample size (250, 500, 1000), test length (30, 60), true trait level (-2.0 to 2.0), and MAT algorithm (MAT, MBAT).
• A three-dimensional trait structure with within- and between-item multidimensionality was simulated.

Main Results:

• MBAT consistently outperformed conventional MAT in reducing bias and mean squared error (MSE) of person parameter estimates.
• MBAT showed particular advantages at the extreme ends of the trait distributions.
• The proposed MBAT approach was found to be computationally feasible and adaptable.

Conclusions:

• The fully Bayesian MBAT approach offers superior accuracy in person parameter estimation compared to conventional MAT.
• MBAT effectively incorporates uncertainty in parameter estimates, addressing limitations of traditional methods.
• The implementation of MBAT in general-purpose software (Stan, R) makes it practical for future research and applications in adaptive testing.