Teen Pregnancy Prevention Evaluation Technical Assistance: A Case Study.

Area of Science:

• Public Health
• Program Evaluation
• Social Science Research

Background:

• The Teen Pregnancy Prevention (TPP) Program funded 65 grantees across two cohorts.
• Grantees were mandated to conduct rigorous evaluations adhering to specific benchmarks.
• Mathematica provided large-scale evaluation technical assistance (ETA) to these grantees.

Purpose of the Study:

• To document Mathematica's experience delivering evaluation technical assistance for the TPP Program.
• To provide an overview of the TPP grant program, its evaluation requirements, and the ETA provided.
• To identify successes, challenges, and lessons learned from the large-scale ETA effort.

Main Methods:

• Case study methodology.
• Analysis of Mathematica's experience providing evaluation technical assistance.
• Review of TPP program structure, evaluation mandates, and grantee outcomes.

Main Results:

• Key lessons learned regarding pre-award considerations for evaluation design.
• Identification of two critical evaluation features (strong counterfactuals, adequate sample sizes) that are difficult to modify via ETA.
• Observation that focusing on rare or unobservable outcomes can limit impact detection.

Conclusions:

• Funders must prioritize robust evaluation designs, including strong counterfactuals and sufficient sample sizes, during the award selection process.
• ETA is less effective in correcting fundamental design flaws present before grant implementation.
• Additional evidence-building activities, such as federally led evaluations and meta-analyses, can supplement grantee findings and address limitations.