When the first try fails: re-implementation of SIMPL in a general surgery residency

Workplace-based assessment (WBA) can facilitate evaluation of operative performance; however, implementation of WBA is sometimes unsuccessful. The American Board of Surgery Entrustable Professional Activities WBA project was launched in July 2023. Some programs will face the challenge of re-implementation of a WBA following previous failures. It is unknown what interventions are most effective for WBA re-implementation. Our goal is to identify barriers and facilitators to re-implementing SIMPL, an operative performance WBA.

The System for Improving and Measuring Procedural Learning (SIMPL) was implemented at our residency in 2018, but usage rates were low. We interviewed residents and faculty to identify barriers to usage and opportunities for improvement. Residents reported that SIMPL usage declined because of several factors, including a low faculty response rate, while some faculty reported not responding because they were unable to login to the app and because usage was not mandated. We then re-implemented SIMPL using a plan based on Kotter's Model of Change. To evaluate impact, we analyzed rates of SIMPL usage when it was first implemented, as well as before and after the date of re-implementation.

In September 2022, we re-implemented SIMPL at our program with measures addressing the identified barriers. We found that, in the six months after re-implementation, an average of 145.8 evaluations were submitted by residents per month, compared with 47 evaluations per month at the start of the original implementation and 5.8 evaluations per month just prior to re-implementation. Faculty completed 60.6% of evaluations and dictated feedback for 59.1% of these evaluations, compared with 69.1% at implementation (44% dictated) and 43% prior to re-implementation (53% dictated).

After identifying barriers to implementation of a WBA, we re-implemented it with significantly higher usage by faculty and residents. Future opportunities exist to implement or re-implement assessment tools within general surgery programs. These opportunities may have a significant impact in the setting of national standardization of workplace-based assessment among general surgery residencies.