Cause, effect and regression in road safety: a case study

Researchers use various ways to determine what change in safety is caused by some treatment. One way is to fit regression equations to cross-section data. Can this work? Another way is to do a before-after study. Is this better? I examine these questions in the setting of a case study. The treated units are rail-highway grade crossings, the treatment is the replacement of 'crossbucks' by 'flashers', and as evidence serve published papers and reports. The results of regression studies are all over the place. Still, one cannot be sure whether this is a sign that the regression failed to capture cause and effect or a sign that the effect of this treatment depends strongly on the conditions in which it is applied. As different regressions use different variables, they cannot corroborate or negate each other's results. This is deeply troubling. The results of before-after studies, in this case, are very consistent. Unfortunately the results do not apply to specific conditions and are therefore of limited practical use. In this respect crash modification functions derived from regressions would have an inherent advantage over those from before-after studies provided they captured cause and effect. There is, at present, little ground for the belief that they do.