Excluding non-English publications from evidence-syntheses did not change conclusions: a meta-epidemiological study

Area of Science:

• Systematic Review Methodology
• Evidence Synthesis
• Health Research

Background:

• Systematic reviews are crucial for evidence synthesis in healthcare.
• Language restrictions in systematic reviews may impact conclusions.
• The effect of excluding non-English publications is not well-established.

Purpose of the Study:

• To assess if limiting systematic reviews to English-language publications alters their conclusions.
• To determine the impact of excluding non-English studies on evidence syntheses.
• To evaluate the viability of language restrictions as a methodological shortcut.

Main Methods:

• Analyzed 59 Cochrane intervention reviews previously unrestricted by language.
• Identified and excluded non-English publications and their associated studies.
• Recalculated meta-analyses for affected outcomes and consulted review authors on conclusion changes.
• Assessed noninferiority based on the proportion of changed conclusions (threshold <10% change).

Main Results:

• 49% of reviews included non-English publications (80 total).
• Exclusion of non-English studies removed 31 studies across 40 outcomes.
• No significant change in effect estimates or statistical significance for 38/40 outcomes.
• Two outcomes showed changed statistical significance, but conclusions remained the same.
• Proportion of changed conclusions was 0.0% (95% CI 0.0-0.6), meeting noninferiority criteria.

Conclusions:

• Excluding non-English publications has a minimal effect on the conclusions of systematic reviews for clinical interventions.
• This exclusion can be a practical methodological shortcut, particularly for rapid review processes.
• The findings support the use of English-language restrictions in certain systematic review contexts.