Language bias in orthodontic systematic reviews: A meta-epidemiological study

Orthodontic systematic reviews (SRs) include studies published mostly in English than non-English languages. Including only English studies in SRs may result in a language bias. This meta-epidemiological study aimed to evaluate the language bias impact on orthodontic SRs.

SRs published in high-impact orthodontic journals between 2017 and 2021 were retrieved through an electronic search of PubMed in June 2022. Additionally, Cochrane oral health group was searched for orthodontic systematic reviews published in the same period.

Study selection and data extraction were performed by two authors. Multivariable logistic regression was implemented to explore the association of including non-English studies with the SRs characteristics. For the meta-epidemiological analysis, one meta-analysis from each SRs with at least three trials, including one non-English trial was extracted. The average difference in SMD was obtained using a random-effects meta-analysis.

174 SRs were included in this study. Almost one-quarter (n = 45/174, 26%) of these SRs included at least one non-English study. The association between SRs characteristics and including non-English studies was not statistically significant except for the restriction on language: the odds of including non-English studies reduced by 89% in SRs with a language restriction (OR: 0.11, 95%CI: 0.01 0.55, P< 0.01). Out of the sample, only fourteen meta-analyses were included in the meta-epidemiological analysis. The meta-epidemiological analysis revealed that non-English studies tended to overestimate the summary SMD by approximately 0.30, but this was not statistically significant when random-effects model was employed due to substantial statistical heterogeneity (ΔSMD = -0.29, 95%CI: -0.63 to 0.05, P = 0.37). As such, the overestimation of meta-analysis results by including non-English studies was statistically non-significant.

Language bias has non-negligible impact on the results of orthodontic SRs. Orthodontic systematic reviews should abstain from language restrictions and use sensitivity analysis to assess the impact of language on the conclusions, as non-English studies may have a lower quality.