Modeling person guessing as a random effect: a Bayesian approach of the two-parameter logistic model

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Guessing behavior in multiple-choice (MC) items is a persistent issue, compromising score validity and interpretability.
• Traditional models often treat guessing as an item-specific parameter, failing to capture individual differences in guessing tendencies.

Purpose of the Study:

• To implement a Bayesian random-effects extension of the two-parameter logistic (2PLE) model.
• To conceptualize and model guessing as a latent individual trait rather than a fixed item parameter.

Main Methods:

• A Monte Carlo simulation study was conducted with a fully crossed design.
• Simulated item response data under the 2PLE model with heterogeneous guessing across various sample sizes (N=100-1,000) and test lengths (6-40 items).
• Bayesian predictive fit indices (LOOIC, WAIC) were used to compare model fit.

Main Results:

• The proposed 2PLE random-effects model demonstrated superior performance compared to the three-parameter logistic (3PL) model.
• Estimates for item discrimination, difficulty, and lower asymptote were more accurate with the 2PLE model, especially under conditions of heterogeneous guessing.
• Bayesian fit indices consistently favored the 2PLE model across all simulated conditions.

Conclusions:

• Reallocating guessing variance from items to individuals within the 2PLE random-effects framework improves psychometric performance.
• Findings support the conceptualization of guessing as a substantive, trait-based process.
• The study underscores the necessity of person-specific guessing models for optimizing inferences from test scores.