Utilizing natural language processing to analyze student narrative reflections for medical curriculum improvement

Medical curricula improvement is an ongoing process to keep material relevant and improve the student's learning experience to better prepare them for patient care. Many programs utilize end-of-year evaluations, but these frequently have low response rates and lack actionable feedback. We hypothesized that student reflections written during a fourth year Sub-Internship could be used retrospectively to mine additional information as feedback for future curriculum adjustments. However, reflections contain a large amount of narrative content that would require a cumbersome and essentially infeasible manual review process for busy medical education faculty.

We developed a Natural Language Processing (NLP) pipeline to automatically identify common themes and topics present in the set of reflective writings that could be used to improve the curriculum. The dataset contains required responses to a faculty issued question submitted between August 2016 and July 2018 about challenges experienced during the medical students fourth year Sub-Internship.

Eleven distinct topics were identified, with several being subsequently addressed in future iterations of the curriculum.

Utilizing NLP on reflective writings was able to identify areas of curriculum improvement, and the NLP results provided a quick and easy way to explore the main themes and challenges expressed by students.