Program report cards: evaluation across multiple residency programs at one institution

The designated institution official (DIO) is responsible for monitoring all residency programs within an institution. Although program-evaluation tools have been developed for residency program directors to use, there are currently no such published evaluation tools for DIOs. This manuscript describes the development and implementation of a standardized, dimensional program report card for the more than 60 residency and fellowship programs at our institution. This report card measures the theoretical construct of residency program performance and is divided into four sections: (1) quality of candidates recruited, (2) the resident educational program, (3) graduate success, and (4) overall house officer satisfaction. Each section is measured by objective and subjective metrics that allow the DIO to record programmatic strengths and weaknesses. These results are confidentially shared with the residency program director and encourage a partnership between the DIO and the program director. It is difficult to provide concrete construct validity with this instrument. The process used to develop the report card seems valid. The authors recognize that this report card is a surrogate for each program's RRC's perception of quality. In the future, the authors hope to work closely with the Accreditation Council for Graduate Medical Education and/or the group on resident affairs of the Association of American Medical Colleges to set national benchmark criteria for acceptable residency program performance for each medical discipline. They hope that DIOs and program directors will be able to compare residency programs objectively and identify areas for improvement at a local and national level.