Detecting Cheating in Large-Scale Assessment: The Transfer of Detectors to New Tests

Area of Science:

• Machine Learning
• Educational Assessment
• Data Science

Background:

• Supervised machine learning models offer high accuracy for detecting test cheaters but require labeled datasets, which are often unavailable early in assessment periods.
• Adapting pre-trained models to new, unlabeled datasets is crucial for early-stage detection when labeled data is scarce.

Purpose of the Study:

• To explore the application of transfer learning for adapting pre-trained cheating detection models to new, unlabeled datasets.
• To evaluate the effectiveness of different transfer learning strategies, including naive transfer and a self-labeling algorithm (SETRED), in improving cheating detection accuracy.

Main Methods:

• Investigated the conditions for successful transfer of cheating detection models between different tests and datasets.
• Evaluated a naive transfer approach (direct reuse of a pre-trained model) and a self-labeling (SETRED) algorithm on an unlabeled dataset.
• Compared the performance of transferred detectors against unsupervised cheating detection methods.

Main Results:

• A transferred cheating detector significantly outperforms unsupervised methods.
• Naive transfer provides a considerable accuracy increase.
• The SETRED algorithm offers a marginal improvement over naive transfer, demonstrating the benefits of adaptive learning.

Conclusions:

• Transfer learning is a viable and effective strategy for enhancing cheating detection accuracy, particularly in early assessment stages.
• Utilizing existing pre-trained cheating detectors can substantially improve detection performance even without new labeled data.
• The findings support the integration of transfer learning techniques to bolster the reliability of automated cheating detection systems.