Predicting failure before it happens: A 5-year, 1042 participant prospective study

Area of Science:

• Medical Education
• Student Engagement
• Academic Performance Prediction

Background:

• Student failure in assessments can lead to significant emotional distress and attrition.
• Early identification of at-risk students is crucial for implementing timely interventions and improving academic success.

Purpose of the Study:

• To develop and validate a predictive model for identifying medical students at risk of academic failure early in their studies.
• To assess the efficacy of simple engagement measures in predicting summative examination performance.

Main Methods:

• A prospective study design was employed, collecting engagement data (formative assessment scores, administrative task compliance, attendance) over the first six weeks of medical school.
• An engagement score was calculated and used to predict performance on a summative examination administered 14 weeks post-commencement.
• Data were collected from five cohorts, totaling 1042 medical students.

Main Results:

• Engagement scores significantly predicted summative examination performance (adjusted R-squared = 0.03, p < 0.001), indicating a small but statistically significant effect size.
• Over 50% of students who ultimately failed the examination had engagement scores in the lowest two deciles.
• Simple linear regression analysis confirmed the predictive power of engagement on student performance.

Conclusions:

• Routinely collected, easily analyzed engagement data can accurately identify at-risk medical students shortly after the commencement of their studies.
• The developed predictive model and toolkit can be adapted for use in similar educational settings to support at-risk students.
• Medical educators should consider the benefits of early detection against the potential ethical implications of using student data for predictive purposes.