The Surgical Clerkship in the COVID Era: A Natural Language Processing and Thematic Analysis

Responses to COVID-19 within medical education prompted significant changes to the surgical clerkship. We analyzed the changes in medical student end of course feedback before and after the COVID-19 outbreak.

Postclerkship surveys from 2017 to 2022 were analyzed including both Likert scale data and free text, excluding the COVID outbreak year 2019-2020. Likert scale questions were compared between pre-COVID (2017-2019) and COVID-era cohorts (2020-2022) with the Mann-Whitney U-test. Free-text comments were analyzed using both thematic analysis and natural language processing including sentiment, word and phrase frequency, and topic modeling.

Of the 483 medical students surveyed from 2017 to 2022, 297 responded (61% response rate) to the included end of clerkship surveys. Most medical students rated the clerkship above average or excellent with no significant difference between the pre-COVID and COVID-era cohorts (70.4% Versus 64.8%, P = 0.35). Perception of grading expectations did significantly differ, 51% of pre-COVID students reported clerkship grading standards were almost always clear compared to 27.5% of COVID-era students (P = 0.01). Pre-COVID cohorts more frequently mentioned learning and feedback while COVID-era cohorts more frequently mentioned case, attending, and expectation. Natural language processing topic modeling and formal thematic analysis identified similar themes: team, time, autonomy, and expectations.

COVID-19 presented many challenges to undergraduate medical education. Despite many changes, there was no significant difference in clerkship satisfaction ratings. Unexpectedly, the greater freedom and autonomy of asynchronous lectures and choice of cases became a highlight of the new curriculum. Future research should investigate if there are similar associations nationally with a multi-institutional study.