Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Explaining variation in the effects of welfare-to-work programs.

David Greenberg1, Robert Meyer, Charles Michalopoulos

  • 1University of Maryland, Baltimore County, USA.

Evaluation Review
|September 10, 2003
PubMed
Summary

Generalizing findings from multisite government training program evaluations is statistically unjustified. A multilevel model reveals that site variations in outcomes require careful analysis, not broad conclusions.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

JAK2<sup>V617F</sup>-positive clonal hematopoiesis in germline BRCA1 versus BRCA2 mutation carriers.

Leukemia·2026
Same author

Asthma-related Pediatric Healthcare Utilization after Hospitalization for RSV and Other Viral Lower Respiratory Tract Infections.

Open forum infectious diseases·2026
Same author

Protective effects of influenza B neuraminidase antibodies against symptomatic influenza virus infection.

The Journal of infectious diseases·2026
Same author

Impact of antenatal biological response modifying drugs on infant infection risk and vaccination rates: a national cohort study.

Archives of disease in childhood·2026
Same author

Oral health after stroke: A mixed-methods study involving healthcare professionals, stroke survivors and primary caregivers.

SAGE open medicine·2026
Same author

Mental health diversion and criminal justice outcomes: a population-based cohort study of 53,605 justice-involved adolescents.

Health & justice·2026

Area of Science:

  • Social policy evaluation
  • Program effectiveness research
  • Multilevel modeling

Background:

  • Government-funded training programs are often evaluated across multiple sites, leading to varied findings.
  • Site-to-site outcome variations are typically linked to program design, participant traits, and local contexts.
  • Narrative synthesis is a common but potentially flawed method for combining multisite evaluation results.

Purpose of the Study:

  • To question the validity of narrative syntheses in multisite program evaluations.
  • To illustrate issues using data from California's Greater Avenues for Independence (GAIN) and the National Evaluation of Welfare-to-Work Strategies (NEWWS).
  • To propose improvements in evaluation strategies for future demonstrations.

Main Methods:

Related Experiment Videos

  • Utilized a simple multilevel evaluation model.
  • Incorporated models for individual outcomes within sites.
  • Analyzed variation in program effects across different sites.

    • Main Results:

    • Generalizations about the effects of the GAIN and NEWWS programs are statistically unsupported.
    • Site-specific variations in outcomes cannot be reliably explained by simple narrative syntheses.
    • The study highlights the limitations of current approaches to multisite evaluation.

    Conclusions:

    • Multisite evaluation findings require rigorous statistical analysis, not just narrative summaries.
    • Future demonstrations could yield better insights into program effect determinants with revised evaluation strategies.
    • Caution is advised when drawing broad conclusions from multisite program evaluations.