A new web-based operative skills assessment tool effectively tracks progression in surgical resident performance

The study aim was to demonstrate that a new database tool for assessment of surgical resident operative skills discerns predictable progression in those skills over successive residency years for specific index case types.

A Web-based interactive database (OpRate) was used to assess selected aspects of resident operative performance as determined by supervising attending surgeons in a medium-sized residency (5-6 residents per postgraduate year [PGY]). This assessment consisted of (1) 3 questions pertaining to patient information, technical, and disease-specific preparedness; (2) 4 laparoscopic technical skills questions pertaining to tissue handling, dexterity, planning, and ability to function independently; and (3) similar open technical skills questions, with the addition of 2 questions defining knot tying ability. Two years of assessment data were examined for cholecystectomy (CH), appendectomy (AP), colon resection (CR), ventral hernia repair (VH), and inguinal hernia repair (IH). Mean scores for total, technical, and preparedness responses, as well as each response area were compared for successive training years for each case type. Mean performance data between postgraduate years were compared by ANOVA, and interitem reliability was assessed by Cronbach's alpha determinations.

OpRate data for 579 cases (142 CH, 67 AP, 73 CR, 202 IH, and 95 VH) were examined. Significant incremental increases in open and laparoscopic technical skills scores by training year were observed for all case types (ANOVA, p < 0.0001). Individual technical skills as well as technical and disease-specific preparedness response areas also demonstrated significant improvement by successive training year. Cronbach's alpha determinations were 0.80-0.94 for the preparedness test items and the skills performance scores for all assessed procedures.

Our early results show that the OpRate assessment tool is effective in identifying expected changes in operative performance across successive training years, with a satisfactory level of internal consistency for the test items. As such, the use of this database tool may offer the opportunity to (1) define performance benchmarks for specific levels of training and (2) identify areas where focused training may be required for specific residents.