From prediction to advising: a multi model approach using NBME subject examinations to inform step 2 CK performance

Following the transition of the USMLE Step 1 to pass/fail scoring, Step 2 Clinical Knowledge (CK) has become a primary metric in residency screening. This study evaluates multiple approaches to identify performance patterns that may support individualised longitudinal advising and targeted preparation aligned with students' educational and specialty goals.

De-identified first-attempt scores from 819 medical students who completed Step 2 CK between May 2021 and April 2026 were analysed. Three models were applied: linear regression, a conditional inference tree, and weighted logistic regression to predict at-risk status. Outcomes included continuous Step 2 CK scores and an at-risk classification defined as scoring at least 1 standard deviation below the national mean for the corresponding passing standard. Model outputs were synthesised to identify performance patterns relevant to advising.

First-attempt pass rates ranged from 95% to 98% across testing years. 184 students (22.4%) met the at-risk criteria. Paediatrics, Internal Medicine, and Neurology showed relatively strong associations with Step 2CK performance. Linear regression explained 71.8% of the variance in Step 2 CK scores. The conditional inference tree produced 17 terminal nodes and yielded a mean root mean squared error of 10.63. Weighted logistic regression demonstrated good discrimination for at-risk classification (AUC = 0.915).

Combining these modelling approaches offers a multidimensional view of student performance that may support targeted preparation for Step 2 CK. By integrating prediction, risk stratification, and performance interpretation, we developed the TRACE Framework to translate institutional assessment data into actionable insights for individualised residency advising.