From field experiments to program implementation: assessing the potential outcomes of an experimental intervention...

Area of Science:

• Social Psychology
• Applied Psychology
• Health Services Research

Background:

• Estimating intervention effectiveness accurately is crucial for public health and policy.
• Traditional analyses often include all randomized individuals, potentially diluting true intervention effects.
• Understanding participant self-selection is key to interpreting intervention outcomes.

Purpose of the Study:

• To compare intervention effect sizes for actual participants versus the entire intervention group.
• To investigate the characteristics and outcomes of non-participants in a preventive intervention study.
• To explore the role of self-selection in intervention engagement and effectiveness.

Main Methods:

• Utilized data from a field experiment testing a preventive intervention for unemployed individuals.
• Applied a procedure to estimate effects on actual participants versus all individuals in the intervention group.
• Conducted comparative analyses of employment and mental health outcomes between participants and non-participants.

Main Results:

• Effect sizes for actual participants were 2-3 times larger than for the entire intervention group.
• Non-participants showed higher reemployment rates post-intervention compared to participants.
• No significant differences were found between participants and non-participants on other measured outcomes.

Conclusions:

• Including non-participants in analyses can significantly underestimate intervention impact.
• Self-selection processes appear to effectively recruit individuals who most need and benefit from the intervention.
• Findings highlight the importance of considering participant engagement and self-selection in intervention research.