The sensitivity of estimates of regression to the mean

Estimations of the effectiveness of remedial treatments in road safety analysis are frequently bedevilled by the problem of regression to the mean (RTM). The number of accidents x observed at a site in the "before" period is a "noisy" quantity: x is Poisson distributed about an (unknown) true mean m for that site, so that x = m + e. Sites selected for treatment tend to have a positive random error component e, which will on average be zero in the "after" period, even if no treatment is applied. Methods for estimating RTM usually require some assumption about the underlying (prior) between-site distribution of the true means f0(m): for example, in the empirical Bayes method, a gamma distribution is assumed. The paper considers the impact of different assumptions for this distribution and, indeed, whether any distributional form needs to be assumed. Using Markov Chain Monte Carlo methods, a variety of distributional forms are assumed for f0(m) and applied to each of a number of real data sets, including that from a major study on the effectiveness of speed cameras. It is shown that, in some cases, the size of the estimated RTM effect can be quite sensitive to the choice of distribution.