Enhancing Clinical Reasoning in Surgical Education: A Randomized Crossover Trial on AI-Generated Individual Feedback...

The positive effect of self-testing has been proven extensively, and past research suggested that feedback could further enhance this effect. This study compared the effectiveness of 2 types of feedback for student entries into an app on learning outcomes regarding clinical reasoning in surgery (RQ1). Additionally, we investigated the effect of general app usage (RQ2) and more intensive app usage (RQ3).

Fourth-year medical students at a German university participated in an entry exam and were then invited to participate in the intervention, which consisted of a self-developed app sending daily notifications with key feature questions throughout the semester, aligned to lecture topics. In a crossover design, every student received either personalized or general feedback. The learning outcome was measured in an exit exam.

There was no difference in exit exam score between the feedback conditions (p = 0. 454). General app usage alone was not associated with a higher exit exam score compared to nonusage, measured through a t-test (p = 0.067), but explorative ANOVAs detected significant interactions between time and app use (p = 0.032). Lastly, we found that higher intensity of app usage was associated with significantly higher scores and therefore a larger learning benefit over time, manifesting in a significant main effect for group (p = 0.003) and a significant interaction effect (p < 0.001).

Our results provide further evidence for the testing effect and demonstrate the successful use of key feature questions in surgical education. Since the results regarding the effect of the feedback types are mixed and the relationship between the constructs seems more complicated, further investigation is needed.