Tackling technical skills competency: a surgical skills rating tool

Area of Science:

• Medical Education
• Surgical Skills Assessment
• Competency-Based Training

Background:

• Accreditation Council for Graduate Medical Education mandates competency-based duty assignments.
• Development of valid and reliable assessment tools is crucial for evaluating surgical competence.
• This study focuses on evaluating a novel rating tool for basic surgical suturing skills.

Purpose of the Study:

• To evaluate the validity and reliability of a rating tool designed to assess basic surgical suturing skills competence.
• To determine if the rating tool can differentiate between varying levels of resident experience.
• To establish the tool's utility in an efficient skills exercise for surgical training.

Main Methods:

• A technical skills exercise involved closing three standardized incisions using different suturing techniques within 15 minutes.
• A 17-item competency marker scale and a 5-point Likert scale global rating were used for evaluation.
• Two independent raters scored video recordings of 12 first-week and 16 final-month postgraduate year 1 surgical residents; statistical analyses included Cohen's kappa and Cronbach's alpha.

Main Results:

• The final-month resident cohort demonstrated significantly higher mean total scores (13.9 vs 11.8, P=0.002) and median global competency ratings (3 vs 1, P<0.001) compared to the first-week cohort.
• Inter-rater reliability, measured by Cohen's kappa, was excellent at 0.77.
• Internal consistency, assessed by Cronbach's alpha, was also excellent at 0.87.

Conclusions:

• The developed rating form is a valuable and efficient tool for assessing technical surgical skill competency.
• Demonstrated construct validity through score improvements with experience.
• The tool exhibits excellent internal consistency and inter-rater reliability, supporting its use in surgical education.