Bayesian model averaging of (a)symmetric item response models in small samples

Area of Science:

• Psychometrics
• Statistical Modeling
• Educational Measurement

Background:

• Asymmetric item response theory (IRT) models offer theoretical advantages but typically demand large sample sizes for reliable parameter estimation.
• Conventional IRT models struggle with stability in small samples due to the complexity of estimating numerous item parameters.
• Newer asymmetric IRT models, such as negative log-log and complementary log-log, provide flexibility in item response function (IRF) shapes and can be applied to smaller datasets.

Purpose of the Study:

• To introduce Bayesian model averaging (BMA) as a method for exploring item asymmetry and estimating IRFs in small samples.
• To evaluate the performance of BMA for item response theory (IRT) analysis compared to traditional model selection and smoothing techniques.
• To demonstrate the feasibility and effectiveness of BMA at both item and test levels for psychometric analysis.

Main Methods:

• Bayesian model averaging (BMA) was applied to a combination of simple symmetric and asymmetric item response theory (IRT) models.
• The proposed BMA methods were tested on an empirical dataset to demonstrate feasibility.
• A simulation study was conducted using small sample sizes (N=100 and N=250) under various complex data-generating conditions.

Main Results:

• The Bayesian model averaging (BMA) approach successfully identified asymmetry present in the data-generating models.
• Across simulation conditions, BMA methods consistently outperformed traditional model selection and kernel smoothing techniques in accuracy.
• The empirical example confirmed the practical applicability of BMA for item response theory (IRT) analysis with limited data.

Conclusions:

• Bayesian model averaging (BMA) provides a robust and practical alternative to complex asymmetric IRT models for small sample sizes.
• The proposed BMA methods offer a flexible approach for exploring item asymmetry and estimating item response functions (IRFs) in psychometric research.
• BMA is a valuable tool for exploratory semi-parametric item response theory (IRT) analysis, especially when dealing with limited data.