A generalized-weights solution to sample overlap in meta-analysis.

Area of Science:

• Statistical methodology
• Meta-analysis
• Econometrics

Background:

• Meta-studies frequently utilize empirical findings from overlapping samples, particularly in fields relying on aggregated observational data.
• This sample overlap, common when multiple estimates derive from the same study or dataset, introduces dependencies that can distort statistical properties.
• Failure to account for sample overlap in meta-analyses can lead to inflated rates of false positives, especially with large meta-sample sizes.

Purpose of the Study:

• To analytically demonstrate how sample overlap inflates false positive rates in meta-analyses.
• To propose a novel meta-estimator, the generalized-weights (GW) meta-estimator, to address the issue of sample overlap.
• To validate the GW meta-estimator's performance through simulations, assessing its ability to control false positives and enhance efficiency.

Main Methods:

• Derivation of analytical conditions under which sample overlap compromises conventional meta-estimators.
• Development of the generalized-weights (GW) meta-estimator, which models the variance-covariance matrix of dependent estimates.
• Construction of the variance-covariance matrix using standard sample size and overlap information from primary studies.

Main Results:

• The GW meta-estimator effectively reduces false positive rates to their nominal levels, mitigating the inflation caused by sample overlap.
• Monte Carlo simulations quantify significant efficiency gains of the GW meta-estimator compared to standard meta-analysis techniques.
• The GW method is adaptable to various effect sizes beyond regression coefficients, including Cohen's d and odds ratios.

Conclusions:

• The generalized-weights (GW) meta-estimator provides a robust solution for meta-analyses with overlapping samples, ensuring statistical validity.
• This method corrects for the correlation structure induced by sample overlap, leading to more reliable research syntheses.
• The GW approach is practical, requiring commonly available data, and offers improved statistical power in meta-analytic research.