Predictive validity of a house-officer selection process at one medical school

This study explored the rank list generated by a process of evaluating applicants for a surgery residency; the process is a revision of an older process (pre-1982) used for that purpose. The study's aim was to learn whether the new process is more predictive of residents' performances in their first year than was the process it replaced and whether the rough rankings computed directly from ratings of applicants are more predictive than the final rankings determined in a meeting by the department faculty, who modify the rough rankings. The 1982-1986 rankings and performances of 32 first-year residents at the University of North Carolina at Chapel Hill School of Medicine were studied and compared with the rankings and performances of 77 applicants from 1976 and 1978-1979 under the previous system. Correlation analyses revealed that for the applicants studied, the rough rankings in the new and the previous systems were equally predictive of performances as measured by rating scales, but that the new system was more predictive of performances on the surgery in-training examination. However, all correlations were small. Finally, the rough rankings obtained under the new system were more predictive of the residents' later performances than were the faculty's final rankings.