Reproducible research practices are underused in systematic reviews of biomedical interventions

Matthew J Page1, Douglas G Altman2, Larissa Shamseer3

  • 1School of Public Health and Preventive Medicine, Monash University, 553 St Kilda Road, Melbourne, Victoria 3004, Australia.

Abstract

Insights

Reproducible research practices are underused in biomedical systematic reviews (SRs). Improving these practices enhances accuracy and allows for reanalysis of SR data.

Area of Science:

  • Biomedical Research
  • Systematic Reviews
  • Meta-Analysis

Background:

  • Reproducible research practices are crucial for verifying scientific findings.
  • Their adoption in systematic reviews (SRs) allows for independent reanalysis and error detection.

Purpose of the Study:

  • To evaluate the utilization of reproducible research practices in biomedical SRs.
  • To identify factors associated with the use of these practices.

Main Methods:

  • A random sample of 110 SRs focusing on therapeutic interventions and published in MEDLINE in February 2014 were analyzed.
  • Data on reproducible research practices were extracted using a 26-item form.
  • Association with Cochrane reviews and PRISMA statement use was explored.

Main Results:

  • Only 65% of SRs provided sufficient detail to recreate all meta-analytic effects.
  • Reproducible research practices were more common in Cochrane reviews (94%) than non-Cochrane reviews (54%).
  • Access to datasets and statistical code was reported in only 30% of SRs.

Conclusions:

  • Reproducible research practices are underutilized in SRs of biomedical interventions.
  • Increased adoption of these practices can improve the reliability and reanalyzability of SR findings.

Related Concept Videos

Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs01:15

Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs

Body:Bioequivalence experimental study designs play a pivotal role in testing the effectiveness of various treatments. Key among these are the repeated measures, cross-over, carry-over, and Latin square designs. In the repeated measures design, each subject receives all treatments, allowing for temporal comparisons. This type of design is useful in reducing variability but requires careful planning to avoid bias.The cross-over design, an economical method, involves sequential administration of...
242
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches01:23

Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches

Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
488
Blind Procedures02:07

Blind Procedures

Ideally, the people who observe and record the children’s behavior are unaware of who was assigned to the experimental or control group, in order to control for experimenter bias. Experimenter bias refers to the possibility that a researcher’s expectations might skew the results of the study. Remember, conducting an experiment requires a lot of planning, and the people involved in the research project have a vested interest in supporting their hypotheses. If the observers knew which...
13.7K
Bioequivalence Experimental Study Designs: Completely Randomized and Randomized Block Designs01:20

Bioequivalence Experimental Study Designs: Completely Randomized and Randomized Block Designs

Body:Bioequivalence experimental study designs are crucial methodologies used in evaluating and comparing the bioavailability of different drug products. These designs are categorized into various types: completely randomized, randomized block, repeated measures, cross and carry-over, and Latin square designs.Completely randomized designs involve randomly allocating treatments to all subjects participating in the experiment. This allocation is achieved by assigning unique random numbers to...
273
Randomized Experiments01:13

Randomized Experiments

The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
9.1K
The Scientific Method in Nursing Process01:18

The Scientific Method in Nursing Process

The scientific method provides the foundation for any research. It is the most reliable and objective of all forms of gaining knowledge and guides in applying research-based evidence in practice and conducting future research.
When using research findings to change practice, one must understand the process used to guide a study. The scientific method is a systematic, step-by-step process that supports the data's validity, reliability, and generalizability. As a result, findings can be...
18.9K