Detection of Item Preknowledge Using Response Times

Area of Science:

• Educational Measurement
• Psychometrics
• Data Science

Background:

• Item preknowledge is a significant form of academic dishonesty in educational assessments.
• Current detection methods primarily rely on item scores, potentially missing subtle indicators of fraud.

Purpose of the Study:

• To introduce a novel statistic for identifying examinees who may have benefited from item preknowledge.
• To evaluate the effectiveness of this new statistic in detecting fraudulent behavior during assessments.

Main Methods:

• Developed a statistic based on the difference in response times between compromised and non-compromised items.
• Proved the statistic's null distribution to be standard normal.
• Conducted simulation studies to assess Type I error rates and statistical power.
• Applied the statistic to a real dataset for practical validation.

Main Results:

• The proposed statistic effectively quantifies response time differences indicative of item preknowledge.
• Simulation results demonstrated acceptable Type I error rates and good power.
• Analysis of real assessment data showed the statistic provides unique information not captured by score-based methods.

Conclusions:

• The new response time-based statistic is a valuable tool for detecting item preknowledge in educational assessments.
• This method complements existing score-based approaches, enhancing the detection of sophisticated cheating.
• Utilizing response time data offers a promising avenue for improving the integrity of educational testing.