Related Experiment Videos
Evaluating resident operative performance: a qualitative analysis of expert opinions
Hilary Sanfey1, Reed G Williams, Xiaodong Chen
1Southern Illinois University School of Medicine, Department of Surgery, Springfield, IL 62794, USA. hsanfey@siumed.edu
Purpose:
To elicit and analyze the strengths and weaknesses of resident operative performance (ROP) as identified by independent expert ratings.
Methods:
Four groups of expert surgeons, blinded to resident training level, evaluated ROP on 2 each of laparoscopic cholecystectomy, thyroidectomy, open inguinal, and laparoscopic ventral hernia repair audio videos, and listed strengths and weaknesses. Comments were coded as technical skills, forward planning, self-direction, situation awareness, and patient safety, and as either procedure specific or general skills.
Results:
Eighteen experts independently entered 1087 comments on 8 ROP strengths and weakness. In the post independent rating discussion, consensus was reached on 85 (28%) of 300 post rating comments with majority agreement on another 83 (28%). Overall, the dominant focus was on forward planning. With the exception of the laparoscopic cholecystectomy cases, raters focused more on general than on procedure-specific skills (P < .05).
Conclusion:
Fewer than 30% of expert rater comments focused on technical skills when considering ROP strengths and weaknesses. Although there was some variation in individual comments, majority agreement was reached on 56% of comments during the post independent rating discussion. These findings will improve rater training and further the implementation of a national assessment process to evaluate end of training surgical competence and operative proficiency.
