Prevalence and Prevention of Reproducibility Deficiencies in Life Sciences Research: Large-Scale Meta-Analyses

BACKGROUND Studies have found that many published life sciences research results are irreproducible. Our goal was to provide comprehensive risk estimates of familiar reproducibility deficiencies to support quality improvement in research. MATERIAL AND METHODS Reports included were peer-reviewed, published between 1980 and 2016, and presented frequency data of basic biomedical research deficiencies. Manual and electronic literature searches were performed in seven bibliographic databases. For deficiency concepts with at least four frequency studies and with a sample size of at least 15 units in each, a meta-analysis was performed. RESULTS Overall, 68 publications met our inclusion criteria. The study identified several major groups of research quality defects: study design, cell lines, statistical analysis, and reporting. In the study design group of 3 deficiencies, missing power calculation was the most frequent (82.3% [95% Confidence Interval (CI): 69.9-94.6]). Among the 6 cell line deficiencies, mixed contamination was the most frequent (22.4% [95% CI: 10.4-34.3]). Among the 3 statistical analysis deficiencies, the use of chi-square test when expected cells frequency was <5 was the most prevalent (15.7% [95% CI: -3.2-34.7]). In the reporting group of 12 deficiencies, failure to state the number of tails was the most frequent (65% [95% CI: 39.3-90.8]). CONCLUSIONS The results of this study could serve as a general reference when consistently measurable sources of deficiencies need to be identified in research quality improvement.