A Web-based survey method for evaluating different components of uncertainty in relative health risk judgments

Uncertainty permeates the process of risk assessment. It arises from recognized sources such as inadequacy of toxicological data, lack of exposure information, and imprecise identification of sensitive populations. In addition to these ambiguities, comparative risk exercises, which amount to risk assessment on a scale wider than that applied to single agents, also entail balancing community values, cost-benefit analyses, and other factors not directly tied to toxicology. Such exercises often convene evaluation panels to attempt a ranking of different stressors or stressor groups. The size of these panels is necessarily limited, and they usually strive to reach some form of consensus on ranking. Because ordinal assignments of risk are so difficult to achieve, they actually evolve into rating agendas in which stressors are categorized as high, medium, or low risks. Whether ranking or rating is its aim, this process, with its emphasis on agreement, usually overlooks two major components of uncertainty. One is variability among raters in assigning a score or category. The other is the degree of uncertainty they implicitly attach to their individual ratings. Both serve as guides to the scope and clarity of the available information. To gather more information about these critical but usually overlooked contributions to uncertainty, and, simultaneously, to query a broader sample of respondents, a survey method was designed to exploit the possibilities of electronic communication based on the World Wide Web. This method can secure risk ratings of selected stressors from many different samples of respondents. In addition, it can also provide information about the extent of ratings variability among risk assessors, individuals, or groups of respondents, about the bases of the ratings, and, concurrently, the confidence they place in their judgment. Comparative risk endeavors conducted in this format make their aims and content easy to modify. Data obtained by such a method can serve as pointers to new research initiatives, to regulator priorities, or to further iterations.