Spurious precision in meta-analysis of observational research

Area of Science:

• Biostatistics
• Epidemiology
• Research Methodology

Background:

• Meta-analysis typically weights studies by inverse-variance for precision.
• Observational study precision can be artificially inflated by methodological choices.
• This spurious precision can interact with publication bias and p-hacking.

Purpose of the Study:

• To demonstrate how spurious precision undermines standard meta-analytic techniques.
• To introduce a novel method, MAIVE, to address this bias.
• To improve the reliability of meta-analyses in observational research.

Main Methods:

• Simulations were conducted to assess the impact of spurious precision.
• Large-scale empirical applications were used for validation.
• The Meta-Analysis Instrumental Variable Estimator (MAIVE) was developed, using sample size as an instrument for reported precision.

Main Results:

• Standard meta-analytic methods, including inverse-variance weighting and funnel plot bias corrections, are undermined by spurious precision.
• Selection models do not fully resolve the issue of biased meta-analyses.
• In some scenarios, a simple unweighted mean of estimates was more accurate than correction methods.
• MAIVE effectively reduces bias by leveraging sample size.

Conclusions:

• Spurious precision is a significant threat to the validity of meta-analyses.
• Existing meta-analytic techniques are insufficient to correct for this bias.
• MAIVE offers a robust and simple solution for enhancing meta-analysis reliability in observational studies.