Comparing Students' Clinical Grades to Scores on a Standardized Patient Note-Writing Task

Area of Science:

• Medical Education
• Clinical Reasoning Assessment
• Healthcare Professional Training

Background:

• Objective structured clinical examinations (OSCEs) offer a platform for real-time documentation of medical students' clinical reasoning.
• The University of Illinois at Chicago College of Medicine (UIC-COM) developed a rubric to score patient notes (PNs) within their OSCEs for senior students.

Purpose of the Study:

• To validate the UIC-COM PN Scoring Rubric as a measure of clinical reasoning.
• To compare PN scores from an OSCE at Columbia University Vagelos College of Physicians and Surgeons (VP&S) with students' clinical rotation performance.

Main Methods:

• A cross-sectional analysis was conducted on 60 third-year medical students at VP&S, randomly selected from different clinical rotation performance tertiles.
• Student PNs were scored using the UIC-COM rubric's four sections: Documentation, Differential Diagnosis, Justification, and Workup, yielding a composite score.
• One-way ANOVA was used to analyze differences in scores across clinical rotation performance tertiles.

Main Results:

• A statistically significant association was found between PN scores and clinical rotation performance.
• Students in higher clinical rotation performance tertiles achieved higher mean Documentation and composite PN scores.
• Specifically, mean composite scores were 61.98, 64.05, and 67.86 for bottom, middle, and top tertiles, respectively (p = 0.02).

Conclusions:

• The study provides validity evidence for using patient note-writing tasks in OSCEs as a measure of clinical reasoning.
• The correlation between PN scores and clinical rotation performance supports the rubric's utility.
• Future research should explore the rubric's application across different learner stages, real-world encounters, and formative assessments.