Building the Building Blocks: Development of a Mission-Driven Residency Application Scoring Rubric for Holistic...

Area of Science:

• Graduate Medical Education
• Medical Education Research
• Residency Admissions

Background:

• Traditional residency application review processes often contain inherent biases.
• There is a lack of literature detailing the creation and utilization of rubrics for holistic review in residency admissions.

Purpose of the Study:

• To describe the design and implementation of a mission-based rubric for holistic review in a residency program.
• To mitigate biases in the residency application review process.

Main Methods:

• Developed a rubric based on the Electronic Residency Application Service (ERAS) application, incorporating domains of experiences, attributes, and metrics.
• Weighted rubric sections based on alignment with the program's mission and values, including personal statements, evaluations, CV data, and academic metrics.
• Applied the rubric to internal medicine-pediatrics residency applications.

Main Results:

• The rubric was found to be clear, efficient, and replicable, with each application taking approximately 15-20 minutes to score.
• Equitable scoring was achieved, with underrepresented in medicine (URM) and non-URM applicants scoring similarly.
• Reproducible results were demonstrated with an intraclass correlation coefficient of 0.82.
• The rubric contributed to an increased number of URM applicants on the initial rank list compared to previous years.

Conclusions:

• Holistic review processes using mission-based rubrics can recruit diverse applicant classes that align with program missions and values.
• Implementing such rubrics helps mitigate biases within graduate medical education systems.
• Larger programs can scale this process by training more reviewers or integrating it at different recruitment stages.