Statistical Heterogeneity in Oral Health Meta-Analyses

Providing the summary effect size and its uncertainty, a prediction interval, and a measure of statistical heterogeneity constitute good reporting practices in meta-analyses. Popular statistical heterogeneity measures comprise the τ² and I² statistics. However, researchers often rely unduly on the I² statistic, using naive categorizations to gauge the extent of heterogeneity, leading to misuses of the meta-analysis models, deficiencies in reporting, and misleading conclusions. The present study aimed to provide empirical evidence on the reporting and interpretation of statistical heterogeneity in systematic reviews of oral health published between 2021 and 2023 in 21 leading specialty and general dental journals. Systematic reviews with at least 1 meta-analysis on binary or continuous outcomes with the most studies were identified. Characteristics were extracted at the systematic review and meta-analysis levels. In total, 313 systematic reviews with meta-analyses were analyzed. Within this cohort of meta-analyses, the random-effects model (89%, n = 278) was frequently applied. Almost all meta-analyses (98%, n = 308) reported the I² value, and 51% (n = 160) reported the τ² value. For this sample, the median I² was 76% (range: 0%-100%), and the median τ² was 0.29 (range: 0-2,632), with 13% (n = 20/160) of these meta-analyses reporting zero τ². Most of the meta-analyses (96%, n = 299) based the heterogeneity interpretation on I² and only 21 (7%) on τ². Although 49% (n = 152) of the meta-analyses chose the meta-analysis model a priori, only 41% (n = 63/152) justified this choice. Furthermore, 42% (n = 131) of the 313 meta-analyses chose the meta-analysis model based on the I². Within oral health meta-analyses, there is evidence of overreliance on I² when reporting and interpreting statistical heterogeneity and selecting the meta-analysis model. The inappropriate use of I² in meta-analysis model selection and interpretation of statistical heterogeneity may have implications for the quality of conclusions delivered to the end users of systematic reviews.