Hawks and Doves: Perceptions and Reality of Faculty Evaluations

Area of Science:

• Medical Education
• Graduate Medical Education
• Internal Medicine Clerkship

Background:

• Internal medicine clerkship grades are crucial for residency selection.
• Inconsistent evaluator ratings can compromise the accuracy and fairness of student performance assessments.
• Clerkship grading committees are recommended, but their precise mechanisms for ensuring accuracy and fairness require further study.

Purpose of the Study:

• To investigate the reliability of assessing and accounting for individual evaluator grading stringency within a medical college grading committee.
• To determine if perceived differences in faculty grading stringency (stringent, lenient, neutral) correlate with actual rating variations.
• To analyze the impact of perceived grading stringency on final grade recommendations.

Main Methods:

• Retrospective analysis of faculty evaluations from a single medical college.
• Faculty were categorized as stringent, lenient, or neutral graders by a grading committee.
• Evaluations were assessed for differences in ratings on specific skills and final grade recommendations using logistic regression.

Main Results:

• "Easy graders" showed a higher probability of awarding above-average ratings, while "hard graders" had a lower probability, though statistical significance was limited to 2 of 8 evaluation questions.
• Odds ratios for higher final suggested grades followed expected patterns (easy/neutral > hard; easy > neutral) but did not reach statistical significance.
• No significant difference was found in final recommended grades between faculty perceived as stringent or lenient.

Conclusions:

• Perceived differences in faculty grading stringency are grounded in reality for specific evaluation elements within clerkships.
• Despite perceptions, the stringency or leniency of graders did not significantly influence the final recommended grades.
• Further research on the "hawk and dove effect" is needed to address grading variations and ensure fairness in student evaluations.