Robustness of meta-analysis results in Cochrane systematic reviews: A case for acupuncture trials

Fragility index (FI) refers to the smallest value that change statistical significance of study results. Meta-analyses of Cochrane systematic reviews are considered as the best evidence for stake holders because they enable effect size estimation that cannot be derived by individual studies, particularly in the field of complementary and integrative medicine (CIM). Thus, this study aimed to evaluate robustness of meta-analysis in Cochrane systematic reviews of acupuncture, one of the most used CIM treatment, using FI of meta-analysis.

Meta-analyses of acupuncture Cochrane systematic reviews with binary benefit outcome measures were searched in PubMed, Embase, and CENTRAL and subject to analysis. Randomized clinical trials (RCTs) or quasi RCTs adopted penetrating needles as treatment and compared with controls such as sham acupuncture, usual care, and active control were included. FI of meta-analyses was calculated in web (https://clinicalepidemio.fr/fragility_ma/); and fragility quotient (FQ) was calculated by dividing FI by total sample size of meta-analysis.

Of 248 retrieved studies, 12 Cochrane systematic reviews with 48 meta-analyses were analyzed. The median FI for statistically significant and non-significant meta-analyses was 7 [interquartile range: 3 - 12] and 6 [interquartile range: 3 - 10]. FQ was similar irrespective of statistical significance. Among controls, meta-analyses adopting sham acupuncture displayed the widest range of FI and FQ.

Robustness of meta-analyses in Cochrane systematic reviews for acupuncture was similar irrespective of statistical significance. Impact of control intervention cannot be determined in this study. As FI enables intuitive interpretation, applications for CIM field can be useful.