Bayesian Checks on Cheating on Tests.

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Current methods for detecting cheating on achievement tests rely on statistical hypothesis testing, often using p-values.
• Existing statistical tests for answer copying present limitations, including arbitrary choices in conditioning on suspected copied responses.
• There is a need for a more flexible and informative approach to quantify the evidence of cheating.

Purpose of the Study:

• To introduce posterior odds as a superior alternative to p-values for statistical hypothesis testing in cheating detection.
• To demonstrate a method for calculating the combinatorial expressions of posterior odds using a reformulated recursive algorithm.
• To show how posterior odds can incorporate prior evidence of cheating, enhancing the detection process.

Main Methods:

• Reformulation of the recursive algorithm for calculating number-correct score distributions.
• Calculation of combinatorial expressions for posterior odds.
• Comparison of posterior odds with traditional p-value approaches in cheating detection models.

Main Results:

• Posterior odds provide a direct measure of the likelihood of cheating, overcoming limitations of p-values.
• The proposed method allows for the incorporation of prior odds, reflecting existing circumstantial evidence of cheating.
• This approach resolves the arbitrary choice between statistical tests that do and do not condition on suspected copied responses.

Conclusions:

• Posterior odds represent a more robust and interpretable metric for detecting cheating in achievement testing.
• The presented calculation method is computationally feasible and integrates seamlessly with existing score distribution algorithms.
• Utilizing posterior odds enhances the ability of testing agencies to make informed decisions regarding suspected academic dishonesty.