Related Experiment Video
Updated: Jul 20, 2025
A Protocol for Using Gene Set Enrichment Analysis to Identify the Appropriate Animal Model for Translational Research
Published on: August 16, 2017
Reducing the false discovery rate of preclinical animal research with Bayesian statistical decision criteria
1Department of Mathematics, University of Siegen, Germany.
Abstract:
The success of preclinical research hinges on exploratory and confirmatory animal studies. Traditional null hypothesis significance testing is a common approach to eliminate the chaff from a collection of drugs, so that only the most promising treatments are funneled through to clinical research phases. Balancing the number of false discoveries and false omissions is an important aspect to consider during this process. In this paper, we compare several preclinical research pipelines, either based on null hypothesis significance testing or based on Bayesian statistical decision criteria. We build on a recently published large-scale meta-analysis of reported effect sizes in preclinical animal research and elicit a non-informative prior distribution under which both approaches are compared. After correcting for publication bias and shrinkage of effect sizes in replication studies, simulations show that (i) a shift towards statistical approaches which explicitly incorporate the minimum clinically important difference reduces the false discovery rate of frequentist approaches and (ii) a shift towards Bayesian statistical decision criteria can improve the reliability of preclinical animal research by reducing the number of false-positive findings. It is shown that these benefits hold while keeping the number of experimental units low which are required for a confirmatory follow-up study. Results show that Bayesian statistical decision criteria can help in improving the reliability of preclinical animal research and should be considered more frequently in practice.
Insights
Bayesian statistical methods improve preclinical research reliability by reducing false positives, outperforming traditional significance testing. This approach enhances drug discovery pipelines while maintaining low experimental unit requirements.
Area of Science:
- Biomedical Research
- Statistical Methodology
- Drug Discovery
Background:
- Preclinical research relies on animal studies for drug development.
- Null hypothesis significance testing (NHST) is standard but can lead to false discoveries.
- Balancing false discoveries and omissions is critical in preclinical pipelines.
Purpose of the Study:
- Compare NHST with Bayesian statistical decision criteria for preclinical research.
- Evaluate methods for improving the reliability of preclinical animal research.
- Assess the impact on false discovery and omission rates.
Main Methods:
- Utilized a large-scale meta-analysis of preclinical effect sizes.
- Employed simulations comparing NHST and Bayesian approaches.
- Incorporated corrections for publication bias and effect size shrinkage.
- Elicited a non-informative prior distribution for comparison.
Main Results:
- Statistical approaches incorporating minimum clinically important difference reduce false discovery rates.
- Bayesian decision criteria decrease false-positive findings in preclinical research.
- These benefits are achievable with a low number of experimental units.
Conclusions:
- Bayesian statistical decision criteria enhance the reliability of preclinical animal research.
- Shifting towards Bayesian methods can reduce false positives in drug discovery.
- Bayesian approaches should be more widely adopted in practice.
Related Concept Videos
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
01:28Preclinical Development: Overview
01:13Accuracy and Errors in Hypothesis Testing
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
01:14Decision Making: Traditional Method
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
01:26Drug Discovery: Overview
02:46Genetic Screens
Forward genetic screens
Forward or “classical” genetic screens involve creating random mutations in an organism’s DNA using radiation, mutagens, or insertion of additional bases, which...