Enhancing Effort-Moderated Item Response Theory Models by Evaluating a Two-Step Estimation Method and...

Area of Science:

• Psychometrics
• Educational Measurement
• Cognitive Psychology

Background:

• Rapid-guessing behavior compromises accurate estimation of item and person parameters in testing.
• Unbiased ability estimates require effective modeling of rapid-guessing patterns.

Purpose of the Study:

• To propose and evaluate alternative modeling approaches for analyzing response data with rapid-guessing patterns.
• To compare the performance of these models against the traditional effort-moderated item response theory (EM-IRT) model.

Main Methods:

• Proposed and evaluated three alternative models: a two-step EM-IRT model and two effort-moderated multidimensional models (EM-MIRT) with between-item and within-item structures.
• Compared parameter recovery accuracy of these models with the traditional EM-IRT model under various simulated conditions.

Main Results:

• The two-step EM-IRT and between-item EM-MIRT models consistently outperformed the traditional EM-IRT model.
• The two-step EM-IRT model demonstrated the best performance, particularly for ability and item difficulty parameter estimation.
• Different rapid-guessing patterns did not impact the two-step EM-IRT model's performance.

Conclusions:

• The two-step EM-IRT model is accurate and efficient for estimating ability in the presence of rapid-guessing responses.
• The between-item EM-MIRT model offers an alternative when no significant mean ability difference exists between rapid-guessers and non-rapid-guessers.