Faculty-Resident Concordance in Workplace-Based Operative Assessment Using the Society for Improving Medical...

Area of Science:

• Medical Education
• Surgical Training
• Urology

Background:

• Assessing operative performance and autonomy in surgical training is crucial.
• The SIMPL assessment tool offers a structured method for evaluating trainees.
• Understanding faculty-trainee agreement is key to refining feedback and improving surgical education.

Purpose of the Study:

• To measure agreement between urology faculty and trainees on operative autonomy and performance.
• To identify facilitators and barriers to using the SIMPL assessment tool in urology residency.
• To assess the feasibility of integrating SIMPL assessments into urology training.

Main Methods:

• Urology faculty and trainees completed SIMPL assessments for operative cases between August 2019 and July 2020.
• Trainee autonomy, performance, and case complexity were measured using paired assessments.
• Faculty-trainee agreement was quantified using weighted kappa coefficients; participant feedback was collected via survey.

Main Results:

• 188 paired SIMPL assessments were analyzed, showing moderate agreement on autonomy (kappa=0.55) and performance (kappa=0.47).
• Agreement was low for case complexity (kappa=0.38), indicating differing perceptions of surgical difficulty.
• Lack of reminders for faculty to complete assessments was a significant barrier, cited by 50% of residents and 37% of faculty.

Conclusions:

• SIMPL assessment is feasible for operative evaluation in urology residency, with moderate agreement on autonomy and performance.
• Lower agreement on case complexity highlights differing perceptions of surgical difficulty between faculty and trainees.
• Addressing barriers, such as implementing a reminder system for assessments, is essential for optimizing the SIMPL tool's utility in surgical training.