Agreement between ranking metrics in network meta-analysis: an empirical study

Area of Science:

• Biostatistics
• Health Research Methodology

Background:

• Network meta-analysis (NMA) synthesizes evidence from multiple randomized controlled trials.
• Ranking metrics are used to order interventions based on efficacy or safety.
• Assessing agreement between different ranking metrics is crucial for reliable NMA interpretation.

Purpose of the Study:

• To empirically evaluate the agreement between treatment hierarchies derived from various ranking metrics in NMA.
• To investigate the influence of network characteristics on the agreement of these hierarchies.

Main Methods:

• Re-analysis of 232 NMA networks (≥4 interventions, 1999-2015).
• Calculated hierarchies using relative treatment effects, probability of best value, and Surface Under the Cumulative Ranking Curve (SUCRA).
• Assessed agreement using Kendall's τ, Spearman's ρ, Yilmaz's τ, and Average Overlap; evaluated network characteristics' impact via variance and sample size metrics.

Main Results:

• High pairwise agreement observed across all ranking metrics.
• Highest agreement between SUCRA and relative treatment effects for both correlation and top-weighted measures.
• Agreement decreased in networks with less precise estimates; probability of best value hierarchies were most sensitive to variance differences.

Conclusions:

• Different ranking metrics in NMA yield similar treatment hierarchies in typical published networks.
• Researchers can choose preferred metrics unless estimates are imprecise or variances are imbalanced.
• In cases of imprecision or imbalance, presenting hierarchies from both probabilistic and non-probabilistic metrics is recommended.