Related Experiment Video
Updated: Jan 3, 2026
Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
P value-driven methods were underpowered to detect publication bias: analysis of Cochrane review meta-analyses
Luis Furuya-Kanamori1, Chang Xu2, Lifeng Lin3
1Research School of Population Health, ANU College of Health and Medicine, Australian National University, Australian Capital Territory, Australia.
Objectives:
The aim of the study was to investigate the effect of number of studies in a meta-analysis on the detection of publication bias using P value-driven methods.
Methods:
The proportion of meta-analyses detected by Egger's, Harbord's, Peters', and Begg's tests to have asymmetry suggestive of publication bias were examined in 5,014 meta-analyses from Cochrane reviews. P values were also assessed in meta-analyses with varying number of studies, whereas symmetry was held constant. A simulation study was conducted to investigate if the above tests underestimate or overestimate the presence of publication bias.
Results:
The proportion of meta-analyses detected as asymmetrical via Egger's, Harbord's, Peters', and Begg's tests decreased by 42.6%, 41.1%, 29.3%, and 28.3%, respectively, when the median number of studies in the meta-analysis decreased from 87 to 14. P values decreased as the number of studies increased in the meta-analysis, despite the level of symmetry remaining constant. The simulation study confirmed that when publication bias is present, P value tests underestimate the presence of publication bias, particularly when study numbers are small.
Conclusion:
P value-based tests used for the detection of publication bias-related asymmetry in meta-analysis require careful examination, as they underestimate asymmetry. Alternative methods not dependent on the number of studies are preferable.
Related Concept Videos
01:09Decision Making: P-value Method
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
01:10Bonferroni Test
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
01:29Bias in Epidemiological Studies
01:14Decision Making: Traditional Method
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
01:24Testing a Claim about Population Proportion
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...