Standardized letter of recommendation for otolaryngology residency selection

Develop a standardized letter of recommendation (SLOR) for otolaryngology residency application that investigates the qualities desired in residents and the letter writer's experience. Compare this SLOR to narrative letters of recommendation (NLORs).

Prospective SLOR/NLOR comparison.

The SLOR was sent to an NLOR writer for each applicant. The applicant's NLOR/SLOR pair was blinded and ranked in seven categories by three reviewers. Inter-rater reliability and NLOR/SLOR rankings were compared. Means of cumulative NLOR and SLOR scores were compared to our departmental rank list.

Thirty-one SLORs (66%) were collected. The SLORs had higher inter-rater reliability for applicant's qualifications for otolaryngology, global assessment, summary statement, and overall letter ranking. Writer's background, comparison to contemporaries/predecessors, and letter review ease had higher inter-rater reliability on the NLORs. Mean SLOR rankings were higher for writer's background (P = .0007), comparison of applicant to contemporaries/predecessors (P = .0031), and letter review ease (P < .0001). Mean SLOR writing time was 4.17 ± 2.18 minutes. Mean ranking time was significantly lower (P < .0001) for the SLORs (39.24 ± 23.45 seconds) compared to the NLORs (70.95 ± 40.14 seconds). Means of cumulative SLOR scores correlated with our rank list (P = .004), whereas means of cumulative NLOR scores did not (P = .18). Means of cumulative NLOR and SLOR scores did not correlate (P = .26).

SLORs require little writing time, save reviewing time, and are easier to review compared to NLORs. Our SLOR had higher inter-rater reliability in four of seven categories and was correlated with our rank list. This tool conveys standardized information in an efficient manner.