Related Experiment Video

Updated: Jul 14, 2025

Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment

Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment

Published on: April 19, 2024

Assessing the magnitude of changes from protocol to publication-a survey on Cochrane and non-Cochrane Systematic

Maximilian Siebert¹, Laura Caquelin², Meisser Madera³

¹Stanford University, Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA.

|October 9, 2023

Abstract

Objective:

To explore differences between published reviews and their respective protocols in a sample of 97 non-Cochrane Systematic Reviews (non-CSRs) and 97 Cochrane Systematic Reviews (CSRs) in terms of PICOS (Patients/Population, Intervention, Comparison/Control, Outcome, Study type) elements and the extent to which they were reported.

Study Design And Setting:

We searched PubMed and Cochrane databases to identify non-CSRs and CSRs that were published in 2018. We then searched for their corresponding Cochrane or PROSPERO protocols. The published reviews were compared to their protocols. The primary outcome was changes from protocol to review in terms of PICOS elements.

Results:

We identified a total of 227 changes from protocol to review in PICOS elements, 1.11 (Standard Deviation (SD), 1.22) changes per review for CSRs and 1.23 (SD, 1.12) for non-CSRs per review. More than half of each sub-sample (54.6% of CSRs and 67.0% of non-CSRs) (Absolute Risk Reduction (ARR) 12.4% [-1.3%; 26.0%]) had changes in PICOS elements. For both subsamples, approximately a third of all changes corresponded to changes related to primary outcomes. Marked differences were found between the sub-samples for the reporting of changes. 95.8% of the changes in PICOS items were not reported in the non-CSRs compared to 42.6% in the CSRs (ARR 53.2% [43.2%; 63.2%]).

Conclusion:

CSRs showed better results than non-CSRs in terms of the reporting of changes. Reporting of changes from protocol needs to be promoted and requires general improvement. The limitations of this study lie in its observational design. Registration: https://osf.io/6j8gd/.

Keywords:

Cochrane Pre-registration Protocols Reporting bias Systematic reviews

More Related Videos

Comparing Bibliometric Analysis Using PubMed, Scopus, and Web of Science Databases

Comparing Bibliometric Analysis Using PubMed, Scopus, and Web of Science Databases

Published on: October 24, 2019

The Participant-Reported Implementation Update and Score PRIUS: A Novel Method for Capturing Implementation-Related Data Over Time

The Participant-Reported Implementation Update and Score PRIUS: A Novel Method for Capturing Implementation-Related Data Over Time

Published on: February 19, 2021

Related Concept Videos

Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches

Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches

Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...

Systematic Error: Methodological and Sampling Errors

Systematic Error: Methodological and Sampling Errors

In the case of systematic errors, the sources can be identified, and the errors can be subsequently minimized by addressing these sources. According to the source, systematic errors can be divided into sampling, instrumental, methodological, and personal errors.
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...

Pharmacovigilance

Pharmacovigilance

Post-marketing surveillance is a critical component of pharmaceutical regulation, often uncovering unanticipated adverse drug reactions (ADRs) once a drug is widely used over an extended period.
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...

Hazard Ratio

Hazard Ratio

The hazard ratio (HR) is a widely used measure in clinical trials to compare the risk of events, such as death or disease recurrence, between two groups over time. It reflects the ratio of hazard rates—the instantaneous risk of the event occurring—between a treatment group and a control group. This measure provides valuable insights into the relative effectiveness of a treatment by assessing how the risk of an event differs between the two groups.
For example, in a clinical trial...

Cochran's Q Test

Cochran's Q Test

Cochran's Q Test is a nonparametric statistical test used to determine if there are potential differences in the outcomes of three or more related groups on a binary (yes/no) or dichotomous outcome. It is essentially an extension of the McNemar Test, which is limited to two related samples - Cochran's Q test can handle three or more related samples, making it more versatile in scenarios where subjects are measured under multiple conditions. The test statistic follows a Chi-Square...

Strategies for Assessing and Addressing Confounding

Strategies for Assessing and Addressing Confounding

Confounding is a critical issue in epidemiological studies, often leading to misleading conclusions about associations between exposures and outcomes. It occurs when the relationship between the exposure and the outcome is mixed with the effects of other factors that influence the outcome. Given that, addressing confounding is of high importance for drawing accurate inferences in research.
Confounding can be addressed at both the design phase of a study and through analytical methods after data...

Articles linked to this work by shared authors, journal, and citation graph.

A Re-Appraisal of Three Network Meta-Analyses to Explain the Discrepancy in Findings for the Efficacy of Fluoxetine for the Treatment of Depression in Children and Adolescents.

Cochrane evidence synthesis and methods·2026

Core open science practices in major medical journals: development of an automatized tool based on a Large Language Model.

Open research Europe·2026

Exploratory Insights into Barriers and Open Practices for Computational Reproducibility in Scientific Research.

F1000Research·2026

Open Science Policies and Practices Among Medical Journals.

JAMA network open·2026

TrialScout links published results to trial registrations using a large language model.

Journal of clinical epidemiology·2026

Can fluoxetine's diminished efficacy in pediatric depression be explained by study sites, baseline severity, age, and psychological interventions? A secondary exploratory meta-regression.

Journal of psychiatric research·2026

See all related articles