Selection effects and prevention program outcomes.

Area of Science:

• Prevention Science
• Causal Inference
• Health Services Research

Background:

• Nonexperimental research designs are common in prevention science but often lack strong causal inference.
• Selection bias, including self-selection and differential attrition, can significantly distort intervention outcome estimates.
• The counterfactual framework offers methods to address these biases in uncontrolled settings.

Purpose of the Study:

• To demonstrate an evaluation design and analytic method for strengthening causal inference in nonexperimental prevention research.
• To apply this method to a multisite study evaluating short-term outcomes of a preventive intervention.
• To account for self-selection and differential dropout biases.

Main Methods:

• Utilized the counterfactual (potential outcomes) model framework.
• Employed methods such as propensity score matching and Heckman two-step approach.
• Applied a bivariate/trivariate probit model using community-based family intervention data and Healthy Youth Survey data.

Main Results:

• Identified significant effects of participant, program, and community attributes on self-selection into the program and program completion.
• Demonstrated that failure to identify selection effects can lead to inaccurate outcome and public health impact estimations.
• Showcased the utility of the analytic approach in a real-world prevention study.

Conclusions:

• The proposed evaluation design and analytic method enhance causal inference in nonexperimental prevention research.
• Accounting for selection bias is crucial for accurate estimation of intervention effectiveness and public health impact.
• Counterfactual models are promising for evaluating interventions in uncontrolled settings and supporting community-level dissemination.