A Novel Method of Setting Performance Standards in Surgery Using Patient Outcomes

Area of Science:

• Medical Education
• Surgical Skill Assessment
• Patient Outcomes

Background:

• Current procedural skill assessments lack evidence-based, outcome-driven standards.
• This deficiency hinders the use of these evaluations in high-stakes certification and credentialing.

Purpose of the Study:

• To describe a novel, outcome-based standard-setting methodology.
• To differentiate between clinical outcomes rather than arbitrary educational goals.

Main Methods:

• Developed a regression model incorporating surgeon performance and patient factors.
• Utilized a receiver operating characteristic (ROC) curve to establish performance standards.
• Applied the Global Evaluative Assessment of Robotic Skill (GEARS) tool to predict urinary continence post-prostatectomy.

Main Results:

• An ROC curve with an area under the curve of 0.75 was generated.
• A predicted probability of 0.35 (Youden Index) was identified as optimal.
• Determined the specific surgeon performance score linked to a 35% probability of urinary incontinence.

Conclusions:

• The novel method is context, patient, and assessment-specific, addressing heterogeneity in clinical settings.
• This approach enhances the credibility of workplace-based assessments in competency-based medical education.
• Ensures high-stakes evaluations prioritize patient safety and satisfactory clinical outcomes.