Detecting Examinees With Item Preknowledge in Large-Scale Testing Using Extreme Gradient Boosting (XGBoost).

Area of Science:

• Machine Learning
• Educational Measurement
• Data Science

Background:

• Machine learning is increasingly used across various research fields.
• The application of machine learning in detecting testing fraud remains underexplored.
• Identifying individuals with item preknowledge is crucial for maintaining test integrity.

Purpose of the Study:

• To provide a technical review of the Extreme Gradient Boosting (XGBoost) algorithm.
• To investigate the effectiveness of XGBoost in identifying examinees with potential item preknowledge.
• To compare XGBoost performance against traditional person-fit statistics.

Main Methods:

• Trained four XGBoost models using distinct feature sets: dichotomous responses, nominal responses, responses with time data, and nominal responses with time data.
• Evaluated model performance using the area under the receiver operating characteristic curve (AUC) and classification metrics (e.g., false-positive rate, true-positive rate, precision).
• Utilized a real dataset comprising both honest test-takers and those exhibiting fraudulent behavior, including prior access to test content.

Main Results:

• XGBoost demonstrated successful classification of both honest and fraudulent test takers.
• Models incorporating response time data alongside item responses achieved notably good classification performance.
• The algorithm effectively identified examinees with item preknowledge.

Conclusions:

• Extreme Gradient Boosting (XGBoost) is a viable and effective tool for detecting testing fraud, particularly item preknowledge.
• Integrating response time data significantly enhances the accuracy of fraud detection models.
• XGBoost offers a promising alternative to traditional methods for safeguarding test security.