The development and implementation of a health-system-wide evaluation system for education activities: build it and...

Academic health centers (AHCs) use education evaluation data for multiple purposes, and they also use multiple methods to collect data in an effort to evaluate the quality of student and faculty performance. Collecting evaluation data in a standardized manner enabling collation and subsequent assessment and interpretation is critically important if the information is to be maximally useful. A case study is presented of PENN Medicine's education evaluation program and the complicated mission of developing a multiprogram, multipurpose evaluation system, developed and implemented from 2003 to 2007. The proposed solution is generalizable to other comparable AHCs. The article begins with a structured analysis of needs, continues with a description of the conceptual evaluation model guiding the system, and offers a summary of the amounts and types of data collected in the years leading to full implementation. The article concludes with a brief list of needs that emerged during implementation and suggestions for future growth. The resulting system is described as supporting the evaluation of clinical teaching of more than 1,200 clinical faculty, students, residents, and fellows across 18 clinical departments with a common set of items. For the 2006-2007 academic year, more than 30,000 faculty evaluations were collected, combined, and then presented in a Web-based teaching dossier. A by-product of this effort was the creation of an ever-expanding data set that supports medical education research.