Modeling Test-Taking Non-effort in MIRT Models.

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Test score validity is threatened by unaddressed examinee non-effort.
• Non-effortful responses can bias parameter estimates in traditional models.
• Integrating effort indicators into measurement models is a potential solution.

Purpose of the Study:

• To propose a multidimensional item response theory (MIRT) method for handling non-effortful responses.
• To evaluate the performance of the MIRT method against existing models.
• To examine the impact of non-effortful responses on item and ability parameter estimates.

Main Methods:

• Development and application of a MIRT model for non-ignorable missing responses.
• Two simulation studies to assess parameter estimation accuracy.
• Comparison with the three-parameter logistic (3PL) and effort-moderated models.

Main Results:

• The unidimensional 3PL model showed degraded parameter estimates with increased non-effort.
• The MIRT model achieved comparable item parameter accuracy to the effort-moderated model.
• The MIRT model yielded the most accurate ability estimates when effort and ability were highly correlated.

Conclusions:

• The proposed MIRT method effectively accounts for non-effortful responses in test data.
• This approach offers a viable alternative for improving the validity of test score inferences.
• Further research should explore limitations and expand applications of the MIRT model.