Evaluating generalist education programs: a conceptual framework. Generalist program evaluation working group

This paper provides and applies a conceptual framework and a list of guiding principles for evaluation of generalist education programs. Programs are systematic efforts to achieve specified objectives. Evaluations gather data in order to improve or appraise programs and have a continuum of purposes and methods. Descriptive evaluations characterize the structures, processes, and outcomes of programs; research evaluations definitively assess the effectiveness of a program in terms of outcomes. Intermediate outcomes are changes in knowledge, attitudes, and skills of program participants; conclusive outcomes reflect the quality of performance of graduates in actual clinical situations. Outcomes are affected by inputs--the qualities of students entering the program. Guiding principles of program evaluation ensure that data gathered are useful. The authors illustrate the guiding principles with an actual pilot study that determined that expert pediatricians, general internists, and family practitioners could agree on key generalist competencies and that explores evaluation design based on these competencies. Finally, they consider the implications of undertaking generalist education evaluation.