Related Experiment Videos
Assessment of Selective Reporting Biases in Studies Included in Campbell Systematic Reviews: A Systematic Review
Julia H Littell1, Jeffrey C Valentine2, Dennis M Gorman3
1Graduate School of Social Work and Social Research, Bryn Mawr College, Bryn Mawr, PA, USA.
Campbell Systematic Reviews
|June 12, 2026
Summary
Recent Campbell systematic reviews show inconsistent appraisal and documentation of reporting biases in studies. Improving these practices is crucial for reliable research synthesis and evidence-based decision-making.
Area of Science:
- Systematic Review Methodology
- Research Integrity
- Evidence Synthesis
Background:
- Critical appraisal of studies is vital for systematic reviews but challenging.
- Risk of bias (ROB) tools aid appraisal, yet reporting bias assessment (outcome reporting bias [ORB] and analysis reporting bias [ARB]) is difficult due to missing study protocols.
Purpose of the Study:
- To identify methods used in recent Campbell systematic reviews to assess reporting bias risks in intervention effect studies.
Main Methods:
- Searched Campbell Library for systematic reviews (SRs) of intervention effects published between 2020 and April 2023.
- Extracted data on reporting bias assessment methods, ROB/quality assessments, and adherence to 12 methodological standards from 49 eligible reviews.
- Data extraction was performed in duplicate, with discrepancies resolved by consensus.
Main Results:
- Reporting bias assessments were diverse and inconsistently documented; over half of reviews lacked explanation/documentation for ORB/ARB.
- Most reviews (40/49) conducted some reporting bias assessment, but results were often unreported.
- None of the reviews met all 12 mandatory methodological standards, failing an average of 4.9 standards.
Conclusions:
- Campbell reviews inconsistently appraise and document reporting biases, hindering reliable interpretation of findings.
- Lack of prospective protocols complicates reporting bias assessment.
- Recommendations and a decision tree are provided to enhance reporting bias assessment in systematic reviews.
Related Concept Videos
01:22Bias
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
01:29Bias in Epidemiological Studies
Biases can arise at various stages of research, from study design and data collection to analysis and interpretation. Recognizing and addressing these biases is essential to ensure the validity and reliability of epidemiological findings.Broadly speaking, biases in epidemiology fall into three main categories: selection bias, information bias, and confounding. A more detailed description of possible biases is:
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
01:31Confirmation Biases
The confirmation bias is the tendency to focus on information that confirms our existing beliefs and ignore information that is inconsistent with our expectations. For example, if you think that your professor is not very nice, you notice all of the instances of rude behavior exhibited by the professor while ignoring the countless pleasant interactions he is involved in on a daily basis. Have you ever fallen prey to the confirmation bias, either as the source or target of such bias?
01:15Systematic Error: Methodological and Sampling Errors
In the case of systematic errors, the sources can be identified, and the errors can be subsequently minimized by addressing these sources. According to the source, systematic errors can be divided into sampling, instrumental, methodological, and personal errors.
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
02:07Blind Procedures
Ideally, the people who observe and record the children’s behavior are unaware of who was assigned to the experimental or control group, in order to control for experimenter bias. Experimenter bias refers to the possibility that a researcher’s expectations might skew the results of the study. Remember, conducting an experiment requires a lot of planning, and the people involved in the research project have a vested interest in supporting their hypotheses. If the observers knew which child was...