The impact of re-engineering a multi-institutional residency program on resident perceptions of the individual...

We report on the development of a survey tool used to assess resident perceptions of support and educational quality among multiple institutions in an integrated surgical residency, as well as its use in measuring the impact of re-engineering on that program.

The University of Connecticut Integrated General Surgical Residency (UCIGSR) is a multisite program that was placed on probation by the Residency Review Committee in Surgery (RRC) in November 1998. This led to a re-evaluation and a re-engineering of the program. In order to better assess the residents' evaluation of the program, we serially examined their attitudes with a survey of 65 questions. A 4-point grading scale (1 = Poor, 4 = Excellent) was used, and each resident was required to complete a survey beginning shortly after RRC probation was announced and at 6-month intervals. Seven global questions in the survey, directed at residency program support and educational quality, were asked for each of the 4 individual adult hospitals, for a total of 28 questions. Statistical analysis of the data was performed using the Jonckhere-Terpstra and the Mann-Whitney U tests.

The results demonstrated significant improvement for all 7 questions in all 4 hospitals between November 1998 (S1) and November 1999 (S3). Average scores for all 7 questions, Hospital Support (HS), Departmental Support (DS), Hospital Teaching (HT), Outpatient Teaching (OTC), Operating Room Teaching (ORT), Grand Rounds (GR), and Morbidity and Mortality Conferences (MM), improved in every hospital by 16-28%. In S1, 1 out of 28 questions received an average score greater than or equal to 3, whereas on the most recent survey, 17 of 28 scored greater than or equal to 3 and 78.5% of the questions demonstrated statistically significant improvement (p < 0.05). Three of the 4 hospitals now have a combined overall average score greater than or equal to 3 for all 7 questions. Areas of strength in each hospital had the least amount of improvement yet remained highly rated.

The survey was able to detect weaknesses and variation in program support and educational quality among institutions in our surgical program. Over time, a re-engineering of the process of educating surgical residents demonstrated a positive effect on all of the institutions. While raising the overall satisfaction level of the residents throughout, the greatest improvement occurred in the lowest rated hospitals. Despite barriers of different institutional cultures and geographic locations, a multi-institutional residency program can institute positive change uniformly, and quantitatively monitor that change.