Does the Medical Student Performance Evaluation Change the Decision to Invite Residency Applicants?

Area of Science:

• Medical Education
• Residency Admissions
• Emergency Medicine

Background:

• Emergency medicine program directors use multiple applicant evaluation sources.
• The Medical Student Performance Evaluation (MSPE) is sometimes used to decide interview offers.
• Prior studies noted variability in MSPE content, but its impact on interview invitations was unstudied.

Purpose of the Study:

• To evaluate how Medical Student Performance Evaluation (MSPE) content affects the likelihood to invite (LTI) emergency medicine residency applicants.
• To test the hypothesis that MSPE review alone rarely alters an applicant's LTI.

Main Methods:

• Prospective observational study of 877 applications to three EM residency programs (2019-2020).
• Reviewers rated LTI (1-5 scale) before and after MSPE review.
• Meaningful LTI change was defined as altering the applicant's interview invitation trajectory.

Main Results:

• The LTI changed ≥1 point for 18.2% of applications.
• A meaningful LTI change occurred in 5.5% of applications (p<0.01).
• Few applicants (0.11%) moved from 'no' to 'yes' LTI; 3.8% moved from 'unsure' to 'yes'; 1.5% moved from 'yes' to 'unsure' or 'no' LTI.

Conclusions:

• MSPE review meaningfully impacted LTI in only 5.5% of emergency medicine residency applications.
• The time spent reviewing variable MSPEs may not be efficient.
• Focusing on objective application components like the Standardized Letter of Evaluation may streamline the review process.