Effect of Grading and Class Rank on Performance in a Surgical Simulation Course

This study seeks to evaluate the psychological and academic effects of grading a medical school simulation course. This study also seeks to evaluate whether a student's class rank affects these results and whether the academic effects of a simulation course persist long-term.

Two separate medical school classes were evaluated. The first class participated in an ungraded surgery simulation course with only formative feedback. The second class participated in a tier-graded course. Qualitative assessment of both courses included daily standardized and self-reported stress surveys, student feedback surveys at the end of the course, and student feedback surveys at the end of the surgery clerkship. Quantitative assessment included pre and postcourse examinations, NBME Surgery Subject Assessment percentile ranks, and COMLEX 2 surgery subscores. Qualitative and quantitative assessments were compared between classes as a whole and between students in the upper and lower half of each class. Results were analyzed using a Student two-tailed t test.

Rocky Vista University College of Osteopathic Medicine Institute of Simulation, Parker, Colorado.

One hundred forty-eight students from the Rocky Vista University College of Osteopathic Medicine Class of 2017 and one hundred fifty-one students from the Rocky Vista University College of Osteopathic Medicine Class of 2018.

Daily standardized stress surveys indicated some increase in stress for students in the graded course, especially for students in the upper half of the graded class. All students adapted to stress significantly by the end of each course, but upper-half students adapted more significantly. Students' self-reported stress levels demonstrated a significant decrease during the ungraded course, but a slight increase during the graded course. Students' self-reported confidence levels rose significantly during the ungraded course, but less significantly during the graded course. Student feedback surveys demonstrated a positive student perception of both courses, but, when stratified, some responses were significantly different between students in the upper and lower half of each class, with upper-half students generally giving a higher rating to the graded course. Clinical knowledge examination scores and improvement in scores within the course were significantly higher for the graded course, with no significant differences between students in the upper and lower half of each class. Students' mean NBME Surgery Subject Assessment percentile rank did not differ significantly between the two classes, but were significantly higher than for previous classes who had not participated in a simulation course. Mean NBME Surgery Subject Assessment percentile rank and COMLEX 2 mean surgery subscore were significantly higher for students in the upper half than for students in the lower half of each class. COMLEX 2 mean surgery subscore did not differ significantly between classes who had participated in a simulation course and previous classes who had not. Students in the upper half of each class in this study performed significantly better than students in the lower half of each class on the mean COMLEX 2 surgery subscore.

This study reaches the following conclusions: (1) objective stress level measurements did not correlate with students' self-reported stress levels; (2) grading a simulation course raises objective stress levels in higher-performing students more than in lower-performing students, but higher-performing students demonstrate the ability to effectively adapt to the increased levels of stress and achieve significant increases in confidence; (3) higher performing students generally demonstrate a greater sense of satisfaction with the simulation course experience when the course is graded; (4) higher-performing and lower-performing students within a class perform in a similar fashion on examinations within the intensely structured environment of a simulation course, whether the course is graded or not; (5) grading a course stimulates all students to perform better on examinations within the course; (6) this higher level of performance and similarity of performance between higher and lower-performing students does not continue when students are not in a structured environment, and students stratify into disparate performance groups in accordance with their class rank; and (7) students exposed to a surgical simulation course perform better on NBME surgery subject examinations than students who have not participated in a simulation course, but this increased level of performance does not extend long-term to national board exams.