Outcomes and sample selection: the case of a homelessness and substance abuse intervention

In the likely event that some clients refuse to participate in a psychosocial field experiment, the estimates of the effects of the experimental treatment on client outcomes may suffer from sample selection bias, regardless of whether the statistical analyses include control variables. This paper explores ways of correcting for this bias with advanced correction strategies, focusing on experiments in which clients refuse assignment into treatment conditions. The sample selection modelling strategy, which is highly recommended but seldom applied to random sample psychosocial experiments, and some alternatives are discussed. Data from an experiment on homelessness and substance abuse are used to compare sample selection, conventional control variable, instrumental variable, and propensity score matching correction strategies. The empirical findings suggest that the sample selection modelling strategy provides reliable estimates of the effects of treatment, that it and some other correction strategies are awkward to apply when there is post-assignment rejection, and that the varying correction strategies provide widely divergent estimates. In light of these findings, researchers might wish regularly to compare estimates across multiple correction strategies.