Is data missing? An assessment of publication bias in orthodontic systematic reviews from 2010 to 2021

To assess the extent of publication bias assessment in systematic reviews (SRs) across the orthodontic literature over the last 12 years and to identify the appropriateness of assessment and association with publication characteristics, including year of publication, journal, searching practices within unpublished literature or attempts to contact primary study authors and others.

We searched six journals and the Cochrane Database of Systematic Reviews for relevant articles, since January 2010, until November 2021. We recorded practices interrelated with publication bias assessment, at the SR and meta-analysis level. These pertained to reporting strategies for searching within unpublished literature, attempts to communicate with authors of primary studies and formal assessment of publication bias either graphically or statistically. Potential associations between publication bias assessment practices with variables such as journal, year, methodologist involvement, and others were sought at the meta-analysis level.

A sum of 289 SRs were ultimately included, with 139 of those incorporating at least one available mathematical synthesis. Efforts to search within unpublished literature were reported in 191 out of 289 Reviews (66.1%), while efforts to communicate with primary study authors were recorded for 150 of 289 of those (51.9%). An appropriate strategy plan to address issues of publication bias, conditional on the number of studies available and the methodology plan reported, was followed in 78 of the 139 meta-analyses (56.1%). Formal publication bias assessment was actually reported in 35 of 139 meta-analyses (25.2%), while only half of those (19/35; 54.3%) followed an appropriately established methodology. Ten of the latter 19 studies detected the presence of publication bias (52.6%). Predictor variables of appropriate publication bias assessment did not reveal any significant effects.

Appropriate methodology and rigorous practices for appraisal of publication bias are underreported in SRs within the orthodontic literature since 2010 and up-to-date, while other established methodologies including search strategies for unpublished data or communication with authors appear currently suboptimal.