Evaluating the effectiveness of air quality regulations: A review of accountability studies and frameworks

Assessments of past environmental policies-termed accountability studies-contribute important information to the decision-making process used to review the efficacy of past policies, and subsequently aid in the development of effective new policies. These studies have used a variety of methods that have achieved varying levels of success at linking improvements in air quality and/or health to regulations. The Health Effects Institute defines the air pollution accountability framework as a chain of events that includes the regulation of interest, air quality, exposure/dose, and health outcomes, and suggests that accountability research should address impacts for each of these linkages. Early accountability studies investigated short-term, local regulatory actions (for example, coal use banned city-wide on a specific date or traffic pattern changes made for Olympic Games). Recent studies assessed regulations implemented over longer time and larger spatial scales. Studies on broader scales require accountability research methods that account for effects of confounding factors that increase over time and space. Improved estimates of appropriate baseline levels (sometimes termed "counterfactual"-the expected state in a scenario without an intervention) that account for confounders and uncertainties at each link in the accountability chain will help estimate causality with greater certainty. In the direct accountability framework, researchers link outcomes with regulations using statistical methods that bypass the link-by-link approach of classical accountability. Direct accountability results and methods complement the classical approach. New studies should take advantage of advanced planning for accountability studies, new data sources (such as satellite measurements), and new statistical methods. Evaluation of new methods and data sources is necessary to improve investigations of long-term regulations, and associated uncertainty should be accounted for at each link to provide a confidence estimate of air quality regulation effectiveness. The final step in any accountability is the comparison of results with the proposed benefits of an air quality policy.

The field of air pollution accountability continues to grow in importance to a number of stakeholders. Two frameworks, the classical accountability chain and direct accountability, have been used to estimate impacts of regulatory actions, and both require careful attention to confounders and uncertainties. Researchers should continue to develop and evaluate both methods as they investigate current and future air pollution regulations.