Accumulation Bias in meta-analysis: the need to consider time in error control

Area of Science:

• Biostatistics
• Meta-analysis methodology
• Scientific research integrity

Background:

• Meta-analyses often rely on accumulated studies, creating time-dependent relationships between analysis timing and study results.
• This temporal dependency introduces "Accumulation Bias," invalidating standard p-value test assumptions and inflating Type I errors.

Purpose of the Study:

• To investigate the influence of time on error control within meta-analysis testing.
• To introduce a framework for modeling dependencies arising from study accumulation and meta-analysis timing.

Main Methods:

• Development of an "Accumulation Bias Framework" to model dependencies in meta-analysis.
• Evaluation of p-value-based tests and likelihood ratio tests under conditions of Accumulation Bias.

Main Results:

• Accumulation Bias is an inevitable consequence of scientific progress and meta-analysis timing.
• P-value-based tests remain invalid even when Accumulation Bias is approximated.
• Likelihood ratio tests demonstrate resilience to Accumulation Bias, providing valid error probability bounds.

Conclusions:

• No p-value-based test can fully overcome Accumulation Bias in meta-analysis.
• Likelihood ratio tests provide a valid approach to error control in time-aware meta-analysis.
• Two strategies for integrating time into error control are proposed: separating or integrating study and meta-analysis timelines using likelihood ratios.