Application of data mining techniques for predicting residents' performance on pre-board examinations: A case study.

Area of Science:

• Medical Education
• Data Mining
• Predictive Analytics

Background:

• Predicting resident academic performance is vital for medical institutions to implement targeted improvement strategies.
• In-training examinations (ITEs) provide valuable data for assessing resident progress.

Purpose of the Study:

• To predict resident performance on preboard examinations using educational data mining (DM) techniques.
• To evaluate the efficacy of different DM algorithms in predicting examination outcomes.

Main Methods:

• A descriptive cross-sectional pilot study involving 841 residents across six specialties.
• Analysis included variance, multiple regression, and three DM algorithms: MLP-ANN, support vector machine, and linear regression.
• Model performance was assessed using Root Mean Square Error (RMSE) and Mean Absolute Error (MAE).

Main Results:

• ITE scores from postgraduate years 2 and 3, and specialty type predicted preboard examination scores (R² = 0.129, P < 0.01).
• MLP-ANN with ten-fold cross-validation demonstrated the best predictive performance (RMSE = 0.325, MAE = 0.212).
• MLP-ANN was used to identify effective association rules for performance prediction.

Conclusions:

• MLP-ANN is a valuable tool for evaluating resident performance on ITEs.
• Enhancing medical educational databases can leverage DM for early identification of at-risk residents.
• Timely, constructive advice can be provided to residents identified through predictive modeling.