Using Item Scores and Distractors to Detect Test Speededness

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Analysis

Background:

• Test speededness, where time limits affect performance, can bias examinee and item parameter estimates.
• Accurate detection of speeded examinees is essential for valid test results.
• Existing methods for detecting speededness have limitations.

Purpose of the Study:

• To develop a novel change-point analysis (CPA) procedure for detecting test speededness.
• To improve the accuracy of identifying speeded examinees and estimating the point at which speededness occurs.
• To leverage information from item distractors, which are commonly available in multiple-choice tests.

Main Methods:

• Development of a new CPA procedure incorporating both item scores and distractor information.
• Extensive simulations to evaluate the performance of the proposed CPA procedure.
• Application of the CPA procedure to a real-world dataset.

Main Results:

• The new CPA procedure demonstrates improved detection of speeded examinees compared to existing methods.
• The procedure yields more accurate change-point estimates.
• Item distractor information significantly contributes to the improved detection of speededness.

Conclusions:

• The proposed CPA procedure offers a more effective method for detecting test speededness.
• Utilizing distractor information in CPA is a valuable approach for improving the accuracy of psychometric analyses.
• This method has practical implications for ensuring the validity of large-scale assessments.