Related Experiment Video
Updated: Jun 29, 2026
Comparing Bibliometric Analysis Using PubMed, Scopus, and Web of Science Databases
Published on: October 24, 2019
High-throughput analysis suggests differences in journal false discovery rate by subject area and impact factor but
L M Hall1, A E Hendricks2,3
1Mathematical and Statistical Sciences, University of Colorado Denver, Denver, CO, USA.
Background:
A low replication rate has been reported in some scientific areas motivating the creation of resource intensive collaborations to estimate the replication rate by repeating individual studies. The substantial resources required by these projects limits the number of studies that can be repeated and consequently the generalizability of the findings. We extend the use of a method from Jager and Leek to estimate the false discovery rate for 94 journals over a 5-year period using p values from over 30,000 abstracts enabling the study of how the false discovery rate varies by journal characteristics.
Results:
We find that the empirical false discovery rate is higher for cancer versus general medicine journals (p = 9.801E-07, 95% CI: 0.045, 0.097; adjusted mean false discovery rate cancer = 0.264 vs. general medicine = 0.194). We also find that false discovery rate is negatively associated with log journal impact factor. A two-fold decrease in journal impact factor is associated with an average increase of 0.020 in FDR (p = 2.545E-04). Conversely, we find no statistically significant evidence of a higher false discovery rate, on average, for Open Access versus closed access journals (p = 0.320, 95% CI - 0.015, 0.046, adjusted mean false discovery rate Open Access = 0.241 vs. closed access = 0.225).
Conclusions:
Our results identify areas of research that may need additional scrutiny and support to facilitate replicable science. Given our publicly available R code and data, others can complete a broad assessment of the empirical false discovery rate across other subject areas and characteristics of published research.
Related Concept Videos
01:21Frequency-dependent Selection
02:02Genomics
02:46Chi-square Analysis
The chi-square test was developed by Pearson in 1990.
The first step of performing a Chi-square analysis is to establish a null hypothesis, which assumes that there is no real...
01:18Patch Clamp
In this method, a glass micropipette containing electrolyte solution is tightly sealed against a small portion of the cell membrane. As a result, a patch of the cell...
01:14Identifying Statistically Significant Differences: The F-Test
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...