Correlation of Ophthalmology Residency Application Characteristics with Subsequent Performance in Residency

Area of Science:

• Medical Education
• Ophthalmology Residency Training
• Surgical Education

Background:

• Identifying successful ophthalmology residency candidates is challenging, especially with the USMLE Step 1 transitioning to Pass/Fail.
• Traditional application metrics may not accurately predict resident performance.
• Artificial intelligence (AI) has the potential to analyze qualitative application components.

Purpose of the Study:

• To identify ophthalmology residency application attributes correlated with successful residency performance.
• To evaluate the utility of AI in analyzing letters of recommendation (LOR), the Dean's letter (MSPE), and personal statements (PS) for predicting performance.

Main Methods:

• Retrospective analysis of 34 application attributes from 55 ophthalmology residents (2004-2019).
• Residents were subjectively ranked into tertiles and deciles by faculty.
• Ophthalmic Knowledge Assessment Program (OKAP) scores served as objective performance metrics; AI analyzed qualitative data.

Main Results:

• Honors in OB/GYN clerkship and a home ophthalmology department correlated with top tertile ranking.
• Core clerkship grades, top 25 USNWR medical school ranking, and PGY-2/PGY-3 OKAP scores predicted top tertile and decile rankings.
• USMLE scores, Alpha-Omega-Alpha (AOA) status, publications, and AI analysis of LORs/MSPE/PS did not correlate with performance.

Conclusions:

• Core clerkship performance and medical school ranking are significant predictors of ophthalmology residency success.
• Objective measures like OKAP scores are associated with high subjective resident rankings.
• Traditional metrics such as USMLE scores, AOA status, and research output showed limited predictive value in this cohort.