Comparing Students' Clinical Grades to Scores on a Standardized Patient Note-Writing Task

Few assessments capture the diagnostic impressions medical students form immediately following patient encounters. However, notes written for objective structured clinical examinations (OSCEs) allow learners to document their clinical reasoning in real time. The University of Illinois at Chicago College of Medicine (UIC-COM) has developed a rubric for scoring patient notes (PNs) in their OSCE for senior students.

To validate the UIC-COM PN Scoring Rubric as a measure of clinical reasoning by comparing PN scores from a similar exam at the Columbia University Vagelos College of Physicians and Surgeons (VP&S) to clinical rotation performance.

Cross-sectional analysis.

From a total of 146 third-year medical students who completed the OSCE at VP&S in spring 2017, we selected 60 at random, 20 from each tertile of clinical rotation performance.

We scored these students' PNs using the rubric's four sections-Documentation, Differential Diagnosis, Justification, and Workup, each scored from 1 to 4-and calculated a composite score (maximum 100). We used one-way ANOVA to examine differences in scores between clinical rotation performance tertiles.

Students in the bottom, middle, and top clinical rotation performance tertiles had mean Documentation scores of 2.54, 2.63, and 2.88, respectively (p = 0.02, bottom vs. top tertile). Mean composite scores were 61.98, 64.05, and 67.86, respectively (p = 0.02, bottom vs. top tertile).

We showed an association between PN scores and clinical rotation performance. Since clinical rotation grades incorporate multiple types of assessments of students' clinical reasoning skills, we believe that this correlation lends validity evidence to using the note-writing task as a measure of clinical reasoning. Future directions include expanding the task to different stages of learners, to real life patient encounters, and to formative rather than summative assessments of note-writing skills.