Random error and undercounting in birth defects surveillance data: implications for inference

There has been an ongoing debate among birth defects investigators about whether or not to publish estimates of rates of birth defects with confidence intervals to allow for comparisons of rates across regions and time. A major impediment in resolving this debate has been the lack of a framework for quantifying uncertainties in the data that can be applied uniformly to birth defects surveillance programs. This report presents an overview of random error and ascertainment bias in birth defects surveillance data, and of the implications of these errors for estimation and comparisons of birth defects rates.

We consider when confidence intervals can be used as part of a strategy to make inference on rates, as well as ratios of or differences between two rates. Worth noting is that confidence intervals only address random error in the data. In the presence of undercounting of cases, estimation of rates and confidence intervals requires knowledge or an estimate of the extent of underascertainment. Rate estimates and confidence intervals that ignore such bias can be misleading. However, if it is reasonable to assume that the ascertainment bias is constant over time (or across regions), then it is possible to make valid comparisons of rates over time (or across regions) using ratio or difference estimators, even when lack of knowledge of the extent of undercounting makes estimating the absolute rate and its confidence interval problematic. Finally, sensitivity analyses can use confidence limits to determine the difference in ascertainment bias necessary to explain an apparent difference in rates.

Because birth defects surveillance systems have evolved in the absence of agreed upon standards to guide the process, it is difficult to determine the extent to which the variability in rates of birth defects across programs or over time is real or due to differences in surveillance methods. Efforts to develop standards for birth defects surveillance may help to minimize the variability in prevalence of birth defects due to differences in case ascertainment methods and allow for evaluations of real temporal and spatial variations in environmental effects. In the meantime, if comparisons of rates need to be made to address public health concerns, it would be prudent to conduct only such comparisons between regions or across time when the degree of case ascertainment can be assumed to be relatively constant across regions and time.