Examination of nonresponse bias in a major residency follow-up study

Nonresponse bias is a potential problem for follow-up studies that track medical school graduates into their residencies. There are two main types of nonresponse: when residents' supervisors do not complete the study evaluation forms and when medical students refuse permission to allow follow-up evaluations during their residencies. The present study discloses the biases that each of these types of nonresponse produced on a set of standard measures of 508 residents from eight 1980s medical school classes at Southern Illinois University. The standard measures (e.g., MCAT scores, grade-point averages, NBME scores) were collected when the residents were medical students. The findings clearly indicate that the scores of the medical students who granted permission for follow-up evaluations had significantly larger means than did the scores on the same measures of those students not granting permission, clearly a source of nonresponse bias. On the other hand, no significant difference in scores (thus negligible bias) was found in cases where supervisors did not complete forms. The authors offer suggestions for medical school policies based on these findings.