Related Experiment Video
Updated: Jun 14, 2026
08:36
The Adjuvant Efficacy of Angong Niuhuang Pill in the Treatment of Viral Encephalitis: A Meta-Analysis of Randomized Controlled Trials
Published on: April 19, 2024
Bias due to changes in specified outcomes during the systematic review process
Jamie J Kirkham1, Doug G Altman, Paula R Williamson
1Centre for Medical Statistics and Health Evaluation, University of Liverpool, Liverpool, England. jjk@liv.ac.uk
Plos One
|March 27, 2010
Summary
Changes to primary outcomes in systematic reviews after protocol publication can introduce bias. Over a fifth of reviews showed discrepancies, with nearly a third of those potentially biased by post-hoc result analysis.
Area of Science:
- Systematic Reviews
- Meta-Analysis
- Research Integrity
Background:
- Modifying outcomes post-protocol publication risks introducing bias.
- Unacknowledged or post-hoc revisions can compromise systematic review integrity.
Purpose of the Study:
- To identify discrepancies between primary outcomes in published systematic review protocols and their corresponding completed reviews.
- To assess the risk of bias in meta-analyses with outcome specification discrepancies.
Main Methods:
- Assessed new systematic reviews from the Cochrane Library.
- Compared primary outcomes listed in protocols versus completed reviews.
- Contacted authors for reasons behind discrepancies and quantified bias risk.
Main Results:
- 22% of protocol/review pairs had outcome discrepancies, with 75% involving primary outcomes.
- Nearly a third of reviews with primary outcome changes showed potential bias from post-hoc analysis.
- Promoted outcomes in reviews were more likely to be statistically significant.
Conclusions:
- Altering outcome importance post-results can lead to biased interpretations.
- Reasons for outcome discrepancies are often unreported, indicating under-recognition of the issue.
- Transparency in reporting outcome specification changes is crucial for systematic review validity.
Related Concept Videos
01:22Bias
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
01:29Bias in Epidemiological Studies
Biases can arise at various stages of research, from study design and data collection to analysis and interpretation. Recognizing and addressing these biases is essential to ensure the validity and reliability of epidemiological findings.Broadly speaking, biases in epidemiology fall into three main categories: selection bias, information bias, and confounding. A more detailed description of possible biases is:
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
02:07Blind Procedures
Ideally, the people who observe and record the children’s behavior are unaware of who was assigned to the experimental or control group, in order to control for experimenter bias. Experimenter bias refers to the possibility that a researcher’s expectations might skew the results of the study. Remember, conducting an experiment requires a lot of planning, and the people involved in the research project have a vested interest in supporting their hypotheses. If the observers knew which child was...
01:20Random and Systematic Errors
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...
01:20Random and Systematic Errors
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...