Controlling Guessing Bias in the Dichotomous Rasch Model Applied to a Large-Scale, Vertically Scaled Testing Program.

Area of Science:

• Educational measurement
• Psychometrics
• Item Response Theory

Background:

• The Rasch model is widely used for educational assessment.
• Guessing on multiple-choice items introduces statistical bias in difficulty estimates.
• This bias can lead to inaccurate interpretations of student proficiency and progress.

Purpose of the Study:

• To demonstrate the elimination of statistical bias in Rasch model difficulty estimates caused by item guessing.
• To analyze the impact of removing guessing bias on scale units and student proficiency estimations.
• To investigate the consequences of uncontrolled guessing bias on measuring student progress over time.

Main Methods:

• Utilizing vertical scaling techniques on a national reading test dataset.
• Applying statistical methods to identify and correct for guessing bias within the Rasch model framework.
• Comparing Rasch model estimates with and without accounting for guessing behavior.

Main Results:

• Eliminating guessing bias results in a nonlinear transformation of the measurement scale.
• The corrected scale disproportionately increases the estimated proficiency of higher-achieving students.
• Failure to control for guessing bias leads to an underestimation of student learning gains over a 7-year period.

Conclusions:

• Controlling for guessing bias is crucial for accurate Rasch modeling in educational assessments.
• The nonlinear scale changes highlight the importance of bias correction for precise proficiency measurement.
• Underestimating student progress due to uncorrected guessing bias has significant educational policy implications.