The measurement of outcomes in the assessment of educational program effectiveness

Postsecondary accrediting agencies recognized by the U.S. Secretary of Education and the Council on Postsecondary Accreditation, including the Liaison Committee on Medical Education (LCME), are required to evaluate educational program effectiveness by determining that institutions and programs document the achievement of their students and graduates in verifiable and consistent ways, indicating that institutional and program purposes are met. For the assessment of medical education programs this represents a departure from the traditional method of inferring quality from institutional compliance with standards for program organization and function. In the new assessment calculus, success is measured as the integrated product of the outcomes, the indicators of achievement that medical schools already are collecting from many sources, for instance, data on premedical achievement and attributes, medical school performance, graduate education ratings and test results, specialty certification, licensure, and practice. Although a recent LCME enquiry showed that 80% of U.S. medical schools were collecting outcome data on students and graduates, there was a lack of coherence and system, little integrated analysis, rare longitudinal study, and limited use of the information to evaluate and revise the curriculum or to validate admissions, promotion, and graduation criteria. The longitudinal study of the quantified results of educational programs need not resurrect old controversies about the linkage between learning in medical school and the quality of doctors' later practice. The purpose of examining outcomes is to gain sharper focus on the achievement of distinctive institutional goals, to facilitate program improvement and renewal, and to better assure the competence of graduates within the boundaries of achievement that schools have drawn as their educational objectives.