Related Experiment Video
Updated: Feb 19, 2026

The Participant-Reported Implementation Update and Score PRIUS: A Novel Method for Capturing Implementation-Related Data Over Time
Published on: February 19, 2021
Reproducible research practices are underused in systematic reviews of biomedical interventions
Matthew J Page1, Douglas G Altman2, Larissa Shamseer3
1School of Public Health and Preventive Medicine, Monash University, 553 St Kilda Road, Melbourne, Victoria 3004, Australia.
Objectives:
To evaluate how often reproducible research practices, which allow others to recreate the findings of studies, given the original data, are used in systematic reviews (SRs) of biomedical research.
Study Design And Setting:
We evaluated a random sample of SRs indexed in MEDLINE during February 2014, which focused on a therapeutic intervention and reported at least one meta-analysis. Data on reproducible research practices in each SR were extracted using a 26-item form by one author, with a 20% random sample extracted in duplicate. We explored whether the use of reproducible research practices was associated with an SR being a Cochrane review, as well as with the reported use of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement.
Results:
We evaluated 110 SRs of therapeutic interventions, 78 (71%) of which were non-Cochrane SRs. Across the SRs, there were 2,139 meta-analytic effects (including subgroup meta-analytic effects and sensitivity analyses), 1,551 (73%) of which were reported in sufficient detail to recreate them. Systematic reviewers reported the data needed to recreate all meta-analytic effects in 72 (65%) SRs only. This percentage was higher in Cochrane than in non-Cochrane SRs (30/32 [94%] vs. 42/78 [54%]; risk ratio 1.74, 95% confidence interval 1.39-2.18). Systematic reviewers who reported imputing, algebraically manipulating, or obtaining some data from the study author/sponsor infrequently stated which specific data were handled in this way. Only 33 (30%) SRs mentioned access to data sets and statistical code used to perform analyses.
Conclusion:
Reproducible research practices are underused in SRs of biomedical interventions. Adoption of such practices facilitates identification of errors and allows the SR data to be reanalyzed.
Insights
Reproducible research practices are underused in biomedical systematic reviews (SRs). Improving these practices enhances accuracy and allows for reanalysis of SR data.
Area of Science:
- Biomedical Research
- Systematic Reviews
- Meta-Analysis
Background:
- Reproducible research practices are crucial for verifying scientific findings.
- Their adoption in systematic reviews (SRs) allows for independent reanalysis and error detection.
Purpose of the Study:
- To evaluate the utilization of reproducible research practices in biomedical SRs.
- To identify factors associated with the use of these practices.
Main Methods:
- A random sample of 110 SRs focusing on therapeutic interventions and published in MEDLINE in February 2014 were analyzed.
- Data on reproducible research practices were extracted using a 26-item form.
- Association with Cochrane reviews and PRISMA statement use was explored.
Main Results:
- Only 65% of SRs provided sufficient detail to recreate all meta-analytic effects.
- Reproducible research practices were more common in Cochrane reviews (94%) than non-Cochrane reviews (54%).
- Access to datasets and statistical code was reported in only 30% of SRs.
Conclusions:
- Reproducible research practices are underutilized in SRs of biomedical interventions.
- Increased adoption of these practices can improve the reliability and reanalyzability of SR findings.
More Related Videos
14:43A Novel Method for Involving Women of Color at High Risk for Preterm Birth in Research Priority Setting
Published on: January 12, 2018
08:29An Open Source Technology Platform to Manufacture Hydrogel-Based 3D Culture Models in an Automated and Standardized Fashion
Published on: March 31, 2022
Related Concept Videos
Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Blind Procedures
Bioequivalence Experimental Study Designs: Completely Randomized and Randomized Block Designs
Randomized Experiments
Simple randomization
Simple...
The Scientific Method in Nursing Process
When using research findings to change practice, one must understand the process used to guide a study. The scientific method is a systematic, step-by-step process that supports the data's validity, reliability, and generalizability. As a result, findings can be...