Related Experiment Videos
What's the meta now? More updates on the problems with systematic reviews
Lesley Uttley1, Yuliang Weng2, Louise Falzon1
1School of Medicine and Population Health, University of Sheffield, Sheffield, United Kingdom.
Background:
Systematic reviews are intended to provide trustworthy evidence synthesis, yet previous iterations of this living review have identified numerous recurring problems in their conduct and reporting. This article presents the third version and second update of the living systematic review examining issues raised across the academic literature.
Methods:
Using consistent eligibility criteria and methods from earlier versions, literature searches were updated to May 2025. Eligible meta-research and editorial articles describing problems with systematic reviews were analyzed to identify emerging themes. Additionally, four basic indicators of methodological quality of the included meta-research were presented across review versions.
Results:
The update included 209 additional articles. Critically low methodological quality and absence of protocols remained among the most frequently reported issues in systematic reviews across disciplines and journals but notably in evidence underpinning clinical practice guidelines. Spin in abstracts and conflicts of interest continued to be common. Apparent improvements in reporting quality were inconsistent, with modest gains in some full-text reporting but persistent deficiencies in abstracts. Authorship diversity of systematic reviews improved in gender representation but remained geographically concentrated in high-income countries, and primary research included in reviews similarly lacked global representativeness. The issue of misalignment between systematic review evidence bases and global burden of disease bring the total number of problems with systematic reviews to 69. Emerging use of automation and artificial intelligence was variably reported. Descriptive comparison of meta-research articles over the three versions of this living review suggests a greater proportion meeting basic quality indicators in more recent updates.
Conclusion:
Across successive updates, problems with systematic reviews remain widespread and consistent rather than isolated. Incremental reporting improvements coexist with persistent concerns about transparency, bias, and representativeness. Future efforts should prioritize evaluating interventions and aligning research incentives to support genuinely trustworthy evidence synthesis.
Related Concept Videos
Systematic Error: Methodological and Sampling Errors
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
Diagnostic and Statistical Manual of Mental Disorders (DSM)
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Bias in Epidemiological Studies
Strategies for Assessing and Addressing Confounding
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
Bias
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...