Detection of risk levels in optical mark recognition exams using categorical feature-based machine learning models

Area of Science:

• Educational Technology
• Data Science
• Machine Learning

Background:

• Optical marking systems are standard for large-scale exam scoring but often miss behavioral data.
• Answer sheet patterns (e.g., blanks, double marks) offer insights into exam security and student conduct.

Purpose of the Study:

• To predict student risk levels using machine learning on optical mark data.
• To assess the value of behavioral patterns in answer sheets for risk evaluation.

Main Methods:

• Utilized categorical features from optical mark data.
• Applied OrdinalEncoder and 5-fold stratified cross-validation.
• Trained and evaluated machine learning models, including CatBoost, against a rule-based baseline.

Main Results:

• CatBoost achieved 73% accuracy and 0.72 weighted F1 score on a five-level risk scale, outperforming the baseline (13% accuracy).
• A three-level risk assessment improved performance to 91% accuracy and 0.85 macro-F1.
• Analyses indicated high discriminative power with minor confusion between adjacent risk levels.

Conclusions:

• Optical mark data contain significant behavioral patterns applicable to risk assessment.
• Machine learning models reliably predict student risk beyond traditional scoring methods.
• This methodology enhances exam security and provides deeper student behavioral insights.