Analytic choices in road safety evaluation: exploring second-best approaches

Area of Science:

• Road safety research
• Transportation engineering
• Statistical modeling

Background:

• Before-and-after studies are crucial for assessing road safety interventions.
• Advanced methods like empirical Bayes (EB) and fully Bayesian techniques require specific data, which may be unavailable or unreliable.
• Researchers face a dilemma: use less robust "second-best" methods or forgo evaluation when data is limited.

Purpose of the Study:

• To investigate how different study design choices impact the estimation of road safety effects.
• To compare the empirical Bayes approach with simpler methods in before-and-after studies.
• To evaluate the effectiveness of "second-best" techniques for controlling confounding factors in road safety evaluations.

Main Methods:

• Analysis of several before-and-after road safety evaluation studies conducted in Norway.
• Comparison of results obtained using the empirical Bayes approach versus simpler methods.
• Assessment of the influence of using or not using a comparison group.
• Examination of the impact of selecting different comparison groups.
• Testing two alternative "second-best" techniques for confounding factor control.

Main Results:

• Study design choices significantly influence the estimated safety effects in before-and-after evaluations.
• The use of simpler approaches, compared to the empirical Bayes method, can lead to different conclusions.
• The inclusion and selection of comparison groups critically affect outcome estimates.
• The "second-best" techniques evaluated did not provide unbiased estimates of safety effects.

Conclusions:

• The choice of methodology in before-and-after road safety studies has a substantial impact on findings.
• "Second-best" methods for controlling confounding factors are not recommended due to their tendency to produce biased results.
• Researchers should prioritize robust data collection and state-of-the-art analytical techniques, such as empirical Bayes, for reliable road safety evaluations.