Evaluating the proper use of weighted mean difference in evidence synthesis: a meta-epidemiology study

The weighted mean difference (WMD) has long been used in evidence syntheses involving continuous outcomes. However, it is sometimes applied interchangeably with the mean difference (MD) despite their conceptual distinction: the WMD is intended solely for pooled estimates across studies, not for individual study results. Misuse of WMD can lead to ambiguity in reporting and interpretation, yet empirical evidence on their prevalence and patterns in published systematic reviews and meta-analyses (SRMAs) remains limited.

We conducted a meta-epidemiological study of SRMAs reporting continuous outcomes, published in The BMJ during 2 periods: 2002-2007 and 2020-2025. Each article was evaluated to determine whether the WMD was reported, whether its use was appropriate, and the rationale for each assessment. Two authors independently performed the evaluations, and any disagreements were resolved through discussion.

In the earlier study period, misuse of the term WMD was widespread, with many SRMAs employing it interchangeably with the MD without clear justification. Over time, the frequency of such misuse decreased, coinciding with the broader adoption of reporting guidelines and increased methodological rigor. Nonetheless, inappropriate applications of WMD persist in more recent publications.

Persistent inconsistencies in WMD use reveal gaps in methodological understanding and reporting practice. Ongoing efforts to refine guidance and promote evidence synthesis literacy are essential to improve the rigor and transparency of SRMAs.