The Predictive Power of Short Answer Questions in Undergraduate Medical Education Progress Difficulty

Area of Science:

• Medical Education Research
• Student Assessment
• Learner Progress Monitoring

Background:

• Investigated the relationship between formative assessments and academic progress in an undergraduate MD program.
• Focused on Short Answer Questions (SAQ) termed Concept Application Exercises (CAE) for assessing knowledge translation.
• CAE scores are formative and not directly used in holistic progress decisions, but students failing objectives are referred to remediation.

Purpose of the Study:

• To determine if Short Answer Questions (SAQ), specifically Concept Application Exercises (CAE), can predict subsequent learner progress difficulties.
• To analyze the predictive power of CAE scores on the need for student remediation.
• To establish the utility of formative SAQs in identifying at-risk medical students.

Main Methods:

• Analyzed data from four medical student cohorts (2022-2025).
• Employed binary logistic regression to model the relationship between CAE score characteristics and remediation committee referral.
• Used remediation committee referral as the dependent variable and CAE scores as independent variables.

Main Results:

• The average CAE score was identified as the strongest predictor of later progress difficulty.
• A one-point decrease in the average CAE score correlated with a 37% increase in the odds of referral to a remediation committee.
• This highlights a significant quantitative link between formative assessment performance and academic progression challenges.

Conclusions:

• Short Answer Questions (SAQ) demonstrate significant predictive value for identifying future academic difficulties in medical learners.
• Concept Application Exercises (CAE) serve as a valuable tool for early identification of students at risk of progress challenges.
• The findings support the integration of formative SAQ data into progress monitoring systems.