Using Deterministic, Gated Item Response Theory Model to detect test cheating due to item compromise

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Item exposure and compromise can lead to score inflation.
• Traditional models may not adequately differentiate between genuine ability and gains from cheating.
• Accurate identification of test cheaters is crucial for maintaining assessment integrity.

Purpose of the Study:

• To propose and validate the Deterministic, Gated Item Response Theory Model (DGM) for detecting examinees who gain scores through item exposure.
• To differentiate between true examinee ability and cheating ability.
• To provide a robust statistical framework for identifying compromised test data.

Main Methods:

• Development of the Deterministic, Gated Item Response Theory Model (DGM).
• Introduction of a "gated" function to decompose examinee performance into true and cheating distributions.
• Application of Hierarchical Markov Chain Monte Carlo for model estimation.
• Validation using a real-world dataset with exposed items.

Main Results:

• The DGM successfully distinguishes between true ability and cheating-related score gains.
• The model effectively identifies examinees with pre-knowledge on exposed items.
• Comparison of true ability and cheating distributions provides a clear indicator of compromised scores.

Conclusions:

• The Deterministic, Gated Item Response Theory Model (DGM) is a viable tool for detecting score gains due to item compromise.
• The model enhances the accuracy of test score interpretation by accounting for potential cheating.
• This approach contributes to more secure and valid standardized testing.