The Quality of Operative Performance Narrative Feedback: A Retrospective Data Comparison Between End of Rotation...

Area of Science:

• Medical Education
• Surgical Training
• Performance Assessment

Background:

• Surgical residency programs utilize end-of-rotation (EOR) evaluations for operative performance feedback.
• Workplace-based assessment tools, such as the System for Improving and Measuring Procedural Learning (SIMPL), offer immediate post-case feedback.
• The comparative quality of narrative feedback from EOR evaluations versus SIMPL is not well-established.

Purpose of the Study:

• To compare the quality of operative performance feedback provided by EOR evaluations and SIMPL in general surgery residency programs.
• To determine which evaluation tool yields more effective feedback for surgical trainees.

Main Methods:

• Collected narrative comments from EOR evaluations and SIMPL assessments across three university-based surgery programs.
• Two independent raters categorized comments on operative skills (specific/general, encouraging/corrective) and assessed their effectiveness (effective, mediocre, ineffective, irrelevant).
• Compared the frequency of effective comments between the two evaluation methods using Chi-square analysis.

Main Results:

• A total of 600 comments were analyzed.
• Effectiveness rates for comments were significantly higher for SIMPL (58.3%) compared to EOR evaluations (10.7%) (P < 0.0001).

Conclusions:

• Workplace-based assessment tools like SIMPL deliver substantially higher quality operative performance feedback than traditional EOR evaluations.
• Implementing SIMPL can improve the effectiveness of feedback in surgical residency training.