A Comparison of Holistic Application Review versus Keyword Search for Residency Application Review

Application review is a lengthy time commitment. The objective of this study is to retrospectively compare the list of recommended applicants as generated by two processes: (1) faculty holistic review and (2) keyword search via Thalamus Cortex, residency application management software, to see how much overlap exists between the two strategies.

Faculty at the training program completed the traditional application review performed by manual, holistic review of each eligible application, and submitted scores on their top 10-15 applicants to the program director (PD). The PD and associate program director (APD) input eight keywords into Cortex and sorted the applications by total keyword count.

There were 86 applicants selected for interview offers by holistic review, the typical process at this program. A second list of 86 applicants was generated by selecting those receiving the highest number of keywords, taking < 1 h to create. Applicants on both lists were compared; the authors found that 17 (19.8%) applicants appeared on both lists. Within the top ten applicants based on keyword count, three (30%) had been identified through holistic review.

Although the keyword process took substantially less time, the percentage agreement is still low. The authors will not move to the keyword method for the next application season; however, they do plan to continue to refine the keywords and compare percent agreement in future years. By closely re-reviewing applicants who score highly on keyword search, this second process can supplement the manual holistic review.