On the Speed Sensitivity Parameter in the Lognormal Model for Response Times and Implications for High-Stakes...

Area of Science:

• Psychometrics
• Educational Measurement
• Cognitive Psychology

Background:

• High-stakes testing often uses multiple test forms with a common time limit.
• Test fairness necessitates that ability estimates are independent of the specific test form administered.
• Differences in speededness between test forms can compromise test fairness and ability estimation comparability.

Purpose of the Study:

• To investigate the impact of speed sensitivity on test form comparability in high-stakes testing.
• To compare the standard lognormal measurement model for response times with an extended version incorporating speed sensitivity.
• To evaluate the effect of differing speed sensitivity across test forms on ability estimates.

Main Methods:

• Comparison of the standard lognormal measurement model (van der Linden) with an extended model (Klein Entink, van der Linden, & Fox) that includes a speed sensitivity parameter.
• Analysis of an empirical dataset to assess model fit.
• Simulation study to examine the influence of varying speed sensitivity on ability estimates.

Main Results:

• The extended lognormal model demonstrated a better fit to the empirical data compared to the model without speed sensitivity parameters.
• Simulations revealed that test forms with different average speed sensitivity produced significantly different ability estimates for slow test takers.
• The disparity in ability estimates was particularly pronounced for high-ability individuals who were slower test takers.

Conclusions:

• The extended lognormal model for response times is recommended for calibrating item pools in high-stakes testing to ensure fairness and accurate ability estimation.
• Accounting for speed sensitivity is crucial for maintaining the comparability of ability estimates across different test forms.
• Further research is needed to address limitations and explore additional research questions related to speededness in testing.