Resolving discrepancies among studies: the influence of dose on effect size

In conducting reviews or meta-analyses, epidemiologists frequently must reconcile conflicting results. This paper addresses heterogeneity in nonexperimental studies. The emphasis is on simple exploratory methods rather than formal approaches. Five examples illustrate how quantitative concordance among studies is possible, even when measured effects appear discrepant. The examples concern ethylene oxide and leukemias, methylene chloride and liver cancer, saccharin and bladder cancer, prenatal lead exposure and birthweight, and aspirin and bleeding tendencies in labor and delivery. Data examined here indicate that differences in dose levels frequently explain heterogeneous effect measures, often outweighing other sources of variability among studies. We present simple methods for combining dose information from the study of interest with dose-response data from other epidemiologic studies or animal studies to derive plausible hypothesized effect levels. These plausible effect sizes are the measures of association that would be predicted, for the actual exposures, by extrapolating from other studies with possibly differing exposure levels. Post hoc power calculations and comparisons of confidence intervals for overlap to reconcile "positive" and "null" studies may be misleading, since these approaches assume a uniform true association obscured by random fluctuations only. Whenever it can be estimated, a plausible effect size should be the starting point to assess findings of either positive or null studies. Without such calculations, comparisons among conflicting studies may not be meaningful.