Den-SOFA: Dental Student Outcome Forecasting Assistant using explainable machine learning models

Area of Science:

• Educational Data Mining
• Artificial Intelligence in Education
• Dental Education

Background:

• AI and ML are increasingly used in dental education for assessment and identifying at-risk students.
• Predictive modeling is challenging due to complex relationships between academic/demographic variables and student achievement.
• Interpretability is crucial for the value of predictive models.

Purpose of the Study:

• To evaluate Den-SOFA, an explainable ML framework, as a proof-of-concept for predicting outcomes on restorative dentistry exit examinations.
• To assess the predictive performance and interpretability of various ML models in dental education.

Main Methods:

• A descriptive-analytical educational data-mining study involving 96 dental students.
• Analysis of 26 academic and demographic variables.
• Evaluation of logistic regression, random forest, XGBoost, CatBoost, and artificial neural networks (ANN) for binary (pass/fail) and multiclass (grade A-F) predictions, using metrics like accuracy, F1-score, and AUC-ROC.
• SHAP analysis for model interpretability.

Main Results:

• ANN achieved the highest discrimination for pass/fail prediction (AUC-ROC=0.906).
• Random forest showed comparable performance with better interpretability.
• Multiclass grade prediction performance was moderate (best AUC-ROC=0.775).
• Academic term, cumulative GPA, phantom lab performance, and theoretical grades were key predictors; demographics had minimal impact.

Conclusions:

• Den-SOFA serves as an exploratory framework for applying explainable ML in dental education.
• Models demonstrated moderate-to-good internal predictive performance, especially for binary outcomes.
• External validation in diverse cohorts is necessary before operational use.