Exploration of heterogeneity in distributed research network drug safety analyses

Distributed data networks representing large diverse populations are an expanding focus of drug safety research. However, interpreting results is difficult when treatment effect estimates vary across datasets (i.e., heterogeneity). In a previous study, risk estimates were generated for selected drugs and potential adverse outcomes. Analyses were replicated across eight distributed data sources using an identical analytic structure. To evaluate heterogeneity of risk estimates across data sources, the estimates were combined with summary-level data characterizing the population of each data source. Meta-analysis, meta-regression, and plots of the influence on overall results versus contribution to heterogeneity were examined and used to illustrate an approach to heterogeneity assessment. Heterogeneity, as measured by the I-squared statistic, was high with variability across outcomes. Plots of the relationship between influence on overall results and contribution to heterogeneity suggest that certain datasets and characteristics were influential but there was variability dependent on the drug and outcome being assessed. Exploratory meta-regression identified many possible influential factors, but may be subject to ecological bias and false positive conclusions. Distributed data network drug safety analyses can produce heterogeneous risk estimates that may not be easily explained. Approaches illustrated here can be useful for research that is subject to similar problems with heterogeneity.