What can we learn from network meta-analyses published in top medical journals

Network meta-analysis (NMA) is a key tool for comparing multiple treatments. However, its reliability can be compromised by inconsistencies in methodology and reporting. We conducted this meta-research study to assess the characteristics and quality of NMAs published between 2019 and 2023 in BMJ, Lancet, JAMA, NEJM, and Annals of Internal Medicine, to identify limitations and propose improvement strategies.

We sampled and evaluated NMAs from the target journals using a 36-item checklist. Reviewers independently extracted data and assessed methodological and reporting quality. We compared our sample's compliance with NMAs from specific domains (Anesthesia, Cancer, Acupuncture, and Cochrane reviews) using odds ratios (ORs) with 95% confidence intervals (CIs) to identify field-specific variations.

Of the 43 included NMAs, the majority originated from Canada (n = 12, 27.91%), China (n = 9, 20.93%), and the UK (n = 9, 20.93%), with 70% published in The BMJ. Overall compliance was high (> 95%) for conducting risk-of-bias assessments, providing a full search strategy, and evaluating inconsistency. However, several key methodological aspects showed suboptimal adherence: statistician involvement (55.81%), transitivity assessment (53.49%), performance of subgroup analyses (53.49%), and meta-regression (44.19%) all had compliance below 60%. Stratified analyses indicated that both the involvement of statistician involvement and the type of intervention significantly influenced methodological and reporting quality. Comparative analyses quantified differences: top medical journals' NMAs were more likely to employ random-effects models than Anesthesia NMAs [OR = 5.49, 95%CI (1.90-15.84)], Cancer NMAs [OR = 10.86, 95%CI (3.95-29.88)], and Cochrane NMAs [OR = 12.35, 95%CI (4.02-37.90)]. They were also more likely to assess consistency compared with Anesthesia NMAs [OR = 47.79, 95%CI (6.18-369.38)], Cancer NMAs [OR = 144.26, 95%CI (18.82-1105.91)], and Cochrane NMAs [OR = 84.00, 95%CI (10.45-675.31)]. Compared with Anesthesia, Acupuncture, and Cochrane NMAs, top medical journal NMAs demonstrated superior methodological rigor and reporting quality, particularly in items related to data collection, presentation of network geometry, and reporting of additional analyses.

Future NMAs should incorporate rational living-update mechanisms and statistical expertise to ensure methodological rigor. Transitivity must be evaluated to confirm study homogeneity, while sensitivity analyses, subgroup analyses, and meta-regression should be systematically conducted to explore heterogeneity. Certainty-of-evidence assessments using standardized frameworks are essential to enhance transparency and to guide evidence-based clinical decision-making and guideline development.