Bounding the effects of social experiments: accounting for attrition in administrative data

Social experiments frequently exploit data from administrative records. However, most administrative data systems are designed to track earnings or benefit payments among residents within a single state. When an experimental participant moves across state lines, his entries in the data system of his state of origin consist entirely of zeros. Such attrition may bias the estimated effect of the experiment.

To estimate the attrition arising from interstate mobility and provide bounds on the effect of the experiment.

Attrition is estimated from runs of zeros at the end of the sample period. Bounds are constructed from these estimates. These estimates can be refined by imposing a stationarity assumption.

The width of the estimated bounds depends importantly on the nature of the data being analyzed. Negatively correlated outcomes provide tighter bounds than positively correlated outcomes.

Attrition can introduce considerable ambiguity into the estimated effects of experimental programs. To reduce ambiguity, one should collect as much data as possible. Even data on outcomes of no direct interest to the objectives of the experiment may be valuable for reducing the ambiguity that arises due to attrition.