How Within-Study Comparisons can Inform Research and Evaluation Design for the Third Sector

Within-study comparisons (WSCs) compare quasi-experimental results to an experimental benchmark and assess the extent to which quasi-experiments (QEs) sacrifice internal validity for external validity. WSCs inform decisions about when to use experiments or QEs, as well as methodological decisions (e.g., covariate selection) about how to implement QEs when they are necessary or preferred. We review the methodological literature on WSCs and present the findings from a WSC of a school voucher program as an example of the use of WSCs to inform policy debates involving the third sector. Prior literature and our results suggest that QE evaluations of third-sector institutions can produce low levels of bias, but only if analytic models include key proxy variables for voluntary self-selection into programs.