Does USMLE Performance Predict Residency Milestone Ratings in Medical Knowledge and Patient Care? A Multi-Site,...

Area of Science:

• Medical Education
• Physician Competency Assessment

Background:

• Medical knowledge acquisition and application are crucial for physician competency.
• Few studies have explored the link between knowledge acquisition and its practical application in residency.

Purpose of the Study:

• To investigate the relationship between medical knowledge acquisition (USMLE scores) and application (ACGME milestones) in a national dataset of medical graduates.

Main Methods:

• Retrospective analysis of data from 3,430 medical school graduates across seven institutions (2016-2018).
• Medical knowledge acquisition assessed via USMLE Step 1 and Step 2 Clinical Knowledge scores.
• Medical knowledge application evaluated using Accreditation Council for Graduate Medical Education (ACGME) milestone ratings during internship, analyzed with mixed-effects regression.

Main Results:

• Failing USMLE Step 1 correlated with higher risk of lower patient care milestone ratings (OR=4.19).
• Failing USMLE Step 2 correlated with higher risk of lower medical knowledge milestone ratings (OR=7.56).
• Higher USMLE scores showed minimal association with higher milestone ratings; no significant link to end-of-year ratings was found.

Conclusions:

• USMLE scores have limited predictive power for the application of medical knowledge in practice.
• USMLE pass/fail status is a more robust predictor of knowledge application.
• USMLE pass/fail status can inform targeted remediation strategies for residents.