The Feasibility of Real-Time Intraoperative Performance Assessment With SIMPL (System for Improving and Measuring...

Area of Science:

• Medical Education
• Surgical Training
• Health Informatics

Background:

• Traditional intraoperative performance assessments for surgical residents face challenges with low participation and delayed feedback.
• There is a growing need for efficient and timely evaluation methods in surgical training programs.

Purpose of the Study:

• To assess the feasibility of integrating a smartphone-based tool, SIMPL (System for Improving and Measuring Procedural Learning), for real-time intraoperative performance assessment in surgical residency programs.
• To evaluate SIMPL's potential to increase the frequency and timeliness of resident evaluations.

Main Methods:

• A multi-center trial involving 15 U.S. general surgery residency programs was conducted between September 2015 and February 2016.
• After a 1-hour training session for 70% of faculty and residents, the SIMPL tool was deployed.
• Feasibility was analyzed using descriptive statistics on training rates, assessment volume, response rates/times, and dictation rates.

Main Results:

• 90% of eligible users completed training, and 13 out of 15 programs adopted SIMPL.
• Over 6,000 assessments were completed by residents and attendings across 3,762 unique operations.
• The system demonstrated increasing usage over time, with a 70% overall response rate and a mean response time of 12 hours.

Conclusions:

• The System for Improving and Measuring Procedural Learning (SIMPL) can be feasibly integrated into surgical training programs.
• SIMPL enhances the frequency and timeliness of intraoperative performance assessments, potentially supporting a competency-based surgical training system.
• Further efforts are needed to address local and systemic challenges for widespread adoption.