Replicating an established open skills curriculum: are the same results obtained in a different setting?

The aim of this study was to ascertain whether the University of Texas, Southwestern (UTSW) open skills curriculum (Goova et al.(13)) could be successfully implemented in the University of Minnesota (UMN) multisite training program. We posed 4 questions: (1) Is the curriculum feasible? (2) Did residents' skills improve? (3) Did UMN residents achieve the same level as UTSW residents? (4) What factors were associated with posttest success?

Postgraduate year-1 residents (n = 22) were enrolled in the curriculum, which included orientation, access to video instruction, 3 months of independent practice using portable kits and practice logbooks, presurveys and postsurveys, and pretesting and posttesting. Evaluation was based on UTSW proficiency measures (time, errors, and total proficiency score). Descriptive statistics, paired sample t tests, analysis of variance, and bivariate correlations were calculated. Results were compared with Goova.

Startup costs at UMN were $9804 vs $776 at UTSW. Our curriculum required 51 direct faculty hours vs 376 at UTSW. UMN trainees' skills improved significantly (mean score = 973 [standard deviation = 267] at baseline vs 1325 [standard deviation = 215] at posttest), but they achieved proficiency in only 38.6% of tasks at posttest, compared with 88.7% by UTSW trainees. Best predictors of UMN posttest proficiency score were (1) categorical vs preliminary resident status (p < 0.001), (2) pretest proficiency score (r = 0.510, p = 0.008), and (3) self-assessed baseline proficiency (r = 0.415, p = 0.027). Participation in skills laboratories during clerkship or fourth year medical school, estimated number of cases (surgeon or first assistant), and number of practice repetitions recorded in booklets were not predictive of the posttest score.

The UTSW open skills curriculum is feasible and effective in a new setting. Differences from UTSW-published success rates may be related to their superior onsite monitoring of practice and a policy requiring residents to achieve proficiency for each task before posttesting.