Heterogeneity in Cochrane and non-Cochrane meta-analyses in orthodontics

Heterogeneity describes the percentage of variability across the study effects that can be attributed to between-study differences in a meta-analysis. The aim of this project was to explore the magnitude of heterogeneity in Cochrane and non-Cochrane meta-analyses in orthodontic research and to identify possible associations between heterogeneity (I²) and a number of study characteristics including number of studies, type of outcome and type of analysis.

The contents of five major orthodontic journals and the Cochrane Database of Systematic Reviews were electronically searched from January 2000 to December 2017 to identify Systematic Reviews (SRs) with at least one meta-analysis. Included records were screened for reporting of I² classified into four categories: 0%, 1-29%, 30-59%, 60-100%. Associations between I² and review-level and synthesis-level characteristics were tested. Univariable and multivariable mixed effects ordinal logistic regression was used to identify significant predictors for statistical heterogeneity.

A total of 72 SRs comprising 391 meta-analyses were included with the majority based on non-Cochrane reviews (n = 54, 75%). Overall, 125 meta-analyses (32%) reported heterogeneity explained by chance (I² = 0%), whereas high values of I² (∼60-100%) were seen in 152 syntheses (39%). In the multivariable analysis, inclusion of each additional study within the synthesis presented 20% higher odds for substantial/considerable heterogeneity compared to lower heterogeneity categories (OR = 1.20; 95%CIs: 1.09, 1.31; p < 0.001). Use of fixed effect analysis (OR = 0.25; 95%CIs: 0.12, 0.55; p = 0.001) was associated with significantly lower odds. Cochrane versus non-Cochrane meta-analyses were not associated with higher odds for substantial/considerable heterogeneity (OR = 2.81; 95%CIs: 0.53, 14.91; p = 0.22).

Substantial statistical heterogeneity is present within a considerable number of orthodontic meta-analyses. Further efforts should be made to improve understanding of decisions to undertake meta-analyses and selection of studies eligible for inclusion.

The consistency of meta-analyses could be improved with more careful consideration of individual study characteristics. Reduced heterogeneity in meta-analyses will ensue more solid evidence based decisions for clinical practice.