The development and validation of a standardized in-training examination for pediatric surgery

This paper reports on the development and validation of a standardized in-training examination for approved graduate program in pediatric surgery. The major purpose of the examination was to provide an external source of data for program and/or educational evaluation. The method used for equating the examination across years, ensuring content relevance are detailed. Given the emphasis on program evaluation, validation procedures concentrated on determining if differences in training would be reflected in performance differences on the examination. The performance of four groups differing in levels of training was compared. Three of the groups represented increasing levels of training within pediatric surgery. The fourth group consisted of trainees in general surgery with an average of 2.5 postgraduate years. Analysis of variance revealed a significant difference in mean performance across and between all four groups. Trend analysis showed the greatest difference to be between trainees in general surgery and trainees just beginning their pediatric surgical training. The smallest difference is noted between the first and second year trainees in pediatric surgery. Performance gains within the second year of training seem to be more specialized and concentrated in certain subject areas. The impact of the above findings and the examination in general for residency program development and evaluation are discussed.