Using an Automated Evaluation System to Improve Medical Student Evaluation Response Rate and Reduce Administrative...

Email reminders facilitate timely task completion but can create substantial administrative burdens for clerkship staff. In our emergency medicine (EM) clerkship, third-year medical students initiate digital evaluation requests after each clinical shift, requiring frequent manual reminders to maintain completion rates. The time-intensive nature of this process highlights the need for a more efficient and scalable approach.

This multi-site, pre-post study evaluated clinical performance evaluation completion rates and time to completion during the required EM clerkship over 48 weeks before and after implementation of an automated reminder system in September 2024. The system integrated existing survey functionality with Microsoft Excel and Microsoft Power Automate to track evaluation requests and completion and generate reminders every 2 days for up to 28 days. Primary outcomes were evaluation completion rate and time to completion. A theoretical cost analysis compared estimated personnel costs of the manual reminder process, based on administrator time and median salary estimates, with projected programming development and maintenance costs over 3 years.

A total of 5747 evaluations were sent during this period to attending and resident evaluators. Overall completion rate rose modestly post-intervention (71.4% to 75.1%, p < 0.001), with attending evaluation rate demonstrating greater improvement from 73.9% to 81.6% (p < 0.001). Conversely, both resident (median 2.67 to 3.98 days) and attending completion time increased (median 2.59 to 2.74 days), p = 0.002. Based on the state salary estimates and our institutional experience, the automated approach could yield estimated net savings of $2850 in the first year and $9560 over 3 years.

Automated reminders improved evaluation completion but did not improve timeliness of completion, which may not translate into meaningful formative benefits if feedback is delayed. Further studies should focus on the impact on cognitive burden and evaluation quality before large-scale automated systems can be optimized and sustainably integrated into medical education.