2020 Rossi Award Lecture: The Evolving Art of Program Evaluation

Area of Science:

• Social program evaluation
• Public policy analysis
• Intervention research

Background:

• Peter Rossi's "Iron Law" suggests zero net impact for social programs, highlighting frequent failures.
• Despite limitations, "failed" or partially successful evaluations offer valuable lessons.
• Advances in study design and statistical methods have improved evaluation analyses.

Discussion:

• Effective public program evaluation increasingly relies on detailed implementation data.
• Linking implementation data (target population, intervention content, delivery process) to impact estimates is crucial for understanding "what works."
• Rossi identified "wrong question, wrong intervention, poor implementation" as key failure reasons.

Key Insights:

• Learning from program evaluation hinges on comprehensive data about intervention implementation.
• Detailed data on program delivery, target demographics, and intervention specifics are essential.
• Analyzing implementation data alongside impact estimates provides deeper insights into program effectiveness.

Outlook:

• Future advancements in social program evaluation depend on integrating implementation data.
• Collaborative efforts between researchers, funders, and program operators are vital for gathering and interpreting implementation data.
• This approach will accelerate understanding of how to best assist vulnerable populations through social programs.