Related Experiment Video
Updated: Jul 30, 2025
08:36
Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment
Published on: April 19, 2024
621
A Note on Cherry-Picking in Meta-Analyses
Daisuke Yoneoka1, Bastian Rieck2
1Center for Surveillance, Immunization, and Epidemiologic Research, National Institute of Infectious Diseases, Tokyo 162-8640, Japan.
Entropy (Basel, Switzerland)
|May 16, 2023
Summary
Researchers can manipulate meta-analysis results by selectively including studies, potentially creating false treatment effects. This selection bias can occur even with standard meta-analysis methods, impacting study reliability.
Area of Science:
- Medical research methodology
- Biostatistics
- Evidence-based medicine
Background:
- Meta-analyses are crucial for synthesizing research findings.
- Selection bias, or cherry-picking, can compromise the integrity of meta-analyses.
- The potential for bias in meta-analysis methodology requires thorough investigation.
Purpose of the Study:
- To theoretically investigate selection bias in meta-analyses.
- To demonstrate how arbitrary inclusion/exclusion criteria can lead to desired results.
- To evaluate the susceptibility of standard meta-analysis methods to cherry-picking.
Main Methods:
- Theoretical modeling of selection bias in meta-analysis.
- Extensive simulation experiments to analyze theoretical findings.
- Evaluation using practical clinical examples.
Main Results:
- Meta-analysts can achieve statistically significant or non-significant results regardless of true treatment effects.
- This manipulation is possible when a sufficient number of studies are available.
- Numerical evaluations confirm that standard meta-analysis methods are vulnerable to cherry-picking.
Conclusions:
- Selection bias poses a significant threat to the validity of meta-analyses.
- Standard meta-analysis techniques may not inherently prevent biased outcomes.
- Researchers must be vigilant against cherry-picking to ensure reliable scientific evidence.
Keywords:
adversarial meta-analysischerry-picking studiesinclusion/exclusion criteriameta-analysisselection biasMore Related Videos
06:26Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI
Published on: November 27, 2019
71.7K
04:34Meta-Analysis of the Effectiveness and Safety of Shugan Jieyu Capsules for the Treatment of Insomnia
Published on: February 17, 2023
1.1K
Related Concept Videos
01:29Bias in Epidemiological Studies
400
Biases can arise at various stages of research, from study design and data collection to analysis and interpretation. Recognizing and addressing these biases is essential to ensure the validity and reliability of epidemiological findings.Broadly speaking, biases in epidemiology fall into three main categories: selection bias, information bias, and confounding. A more detailed description of possible biases is:
400
02:46Chi-square Analysis
38.6K
The chi-square test is a statistical hypothesis test. It is used to check whether there is a significant difference between an expected value and an observed value. In the context of genetics, it enables us to either accept or reject a hypothesis, based on how much the observed values deviate from the expected values.
The chi-square test was developed by Pearson in 1990.
The first step of performing a Chi-square analysis is to establish a null hypothesis, which assumes that there is no real...
The chi-square test was developed by Pearson in 1990.
The first step of performing a Chi-square analysis is to establish a null hypothesis, which assumes that there is no real...
38.6K
01:52Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
01:27Confounding in Epidemiological Studies
209
Confounding in statistical epidemiology represents a pivotal challenge, referring to the distortion in the perceived relationship between an exposure and an outcome due to the presence of a third variable, known as a confounder. This variable is associated with both the exposure and the outcome but is not a direct link in their causal chain. Its presence can lead to erroneous interpretations of the exposure's effect, either exaggerating or underestimating the true association. This...
209
01:25Dose-Response Relationship: Selectivity and Specificity
7.0K
Drugs exert their therapeutic effects by interacting with receptors, enzymes, or ion channels that are present throughout the human body. The strength and duration of the interaction between a drug and its target receptor are characterized by the selectivity and specificity of the drug. Selectivity refers to a drug's strong preference for its intended target over other targets. For instance, isoprenaline, a non-selective β-adrenergic agonist, interacts with both β1- and...
7.0K
01:22Bias
4.3K
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
4.3K