Exploration of the Stacking Ensemble Machine Learning Algorithm for Cheating Detection in Large-Scale Assessment.

Area of Science:

• Educational Measurement
• Computer Science
• Artificial Intelligence

Background:

• Cheating detection in large-scale assessments is a critical area of research.
• Existing studies have not explored stacking ensemble machine learning for this purpose.
• The challenge of class imbalance in cheating detection datasets remains unaddressed.

Purpose of the Study:

• To investigate the efficacy of the stacking ensemble machine learning algorithm for detecting cheating behaviors.
• To compare the performance of stacking with other ensemble and non-ensemble machine learning algorithms.
• To address class imbalance and optimize feature sets for improved detection.

Main Methods:

• Applied stacking ensemble machine learning to analyze item response, response time, and augmented test-taker data.
• Compared stacking with bagging and boosting ensemble methods, and six base machine learning algorithms.
• Utilized resampling techniques to handle class imbalance and evaluated different feature sets.

Main Results:

• Stacking ensemble, resampling, and augmented summary data demonstrated superior performance in cheating detection.
• The stacking meta-model, particularly using Gradient Boosting and Random Forest, achieved the highest accuracy.
• Optimal performance was observed with item responses and augmented summary statistics, using a 10:1 under-sampling ratio.

Conclusions:

• Stacking ensemble machine learning offers a promising approach for robust cheating detection in educational assessments.
• Addressing class imbalance and incorporating augmented data are crucial for enhancing detection accuracy.
• The study provides a validated methodology for improving the integrity of large-scale testing.