Pitfalls in evaluating the surgical student

One hundred five students in our surgical clerkship were evaluated independently by two faculty members, a chief resident, a written multiple-choice examination, and two oral examinations. Faculty and chief resident evaluations included a written explanation and a score from 1 (honors) to 5 (failing). Oral and written examination scores were adjusted to the same numerical scale. Correlation coefficients (r) and probabilities (P) by the t test were calculated to determine validity of the components of the students' grades. In comparison with the oral examination, written examination, and final mean score, the chief residents' evaluations exhibited a validity significantly lower than that of both faculty groups (P less than 0.01). This finding was associated with the following differences in content of associated written evaluations. Residents were more likely to mention "skills" or "technique" (P less than 0.001) and less likely to comment on "logic," "judgment," or "reasoning" (P less than 0.001). Comments concerning "appearance," "enthusiasm," "diligence," and "motivation" were not significantly different among these groups. Experience gained as a member of a surgical teaching faculty or by an affiliation with a teaching program resulted in a greater tendency to focus on thought processes and less emphasis on technical factors in the evaluation of the student on a surgical rotation. Education of surgical house staff must include elucidation of appropriate criteria by which students should be judged. A welcome by-product might be a shift in emphasis in surgical education from a myriad of technical details to a logical approach to diagnosis and treatment.