Information collected during the residency match process does not predict clinical performance

To determine whether information collected during the National Resident Matching Program (NRMP) predicts clinical performance during residency.

Ten faculty members rated the overall quality of 69 pediatric house officers as clinicians. After rating by the faculty, folders were reviewed for absolute rank on the NRMP match list; relative ranking (where they ranked in their postgraduate year 1 [PGY-1] group); scores on part I of the National Board of Medical Examiners (NBME) examination; grades during medical school pediatrics and internal medicine rotations; membership in the Alpha Omega Alpha Medical Honor Society; scores of faculty interviews during intern application; scores on the pediatric in-service examination during PGY-1; and scores on the American Board of Pediatrics certification examination.

There was substantial agreement among faculty raters as to the overall quality of the residents (agreement rate, 0.60; kappa = 0.50; P = .001). There was little correlation between faculty ratings and absolute (r = 0.19; P = .11) or relative (r = 0.20; P = .09) ranking on the NRMP match list. Individuals ranked in the top 10 of the match list had higher faculty ratings than did their peers (mean +/- SD, 3.66+/-1.22 vs. 3.0+/-1.27; P = .03), as did individuals ranked highest in their PGY-1 group (mean +/- SD, 3.88+/-1.45 vs. 3.04+/-1.24; P = .03). There was no correlation between faculty ratings and scores on part I of the NBME examination (r = 0.10; P = .49) or scores on the American Board of Pediatrics certification examination (r = 0.22; P = . 11). There were weak correlations between faculty ratings and scores of faculty interviews during the intern application process (r = 0.27; P = .02) and scores on the pediatric in-service examination during PGY-1 (r = 0.28; P = .02). There was no difference in faculty ratings of residents who were elected to Alpha Omega Alpha during medical school (mean +/- SD, 3.32+/-1.21) as compared with those who were not (mean +/- SD, 3.08+/-1.34) (P = .25).

There is significant agreement among faculty raters about the clinical competence of pediatric residents. Medical school grades, performance on standardized examinations, interviews during the intern application process, and match-list ranking are not predictors of clinical performance during residency.