Clinical trials and statistical verdicts: probable grounds for appeal

Area of Science:

• Biostatistics
• Clinical Trial Analysis
• Medical Research Methodology

Background:

• Conventional clinical trial interpretation heavily relies on the p-value.
• The p-value represents a false-positive rate, not the probability of a hypothesis being correct.
• Existing statistical methods have limitations in accurately reflecting the certainty of study findings.

Purpose of the Study:

• To introduce an extension of Bayes' theorem for analyzing statistical tests in clinical trials.
• To quantify the posterior probability of an investigator's hypothesis being correct.
• To demonstrate the limitations of classic statistical analysis using reanalyzed clinical trial data.

Main Methods:

• Applied an extension of Bayes' theorem to statistical test analysis.
• Reanalyzed several previously published clinical trials using Bayesian methods.
• Compared Bayesian analysis outcomes with conventional p-value interpretations.

Main Results:

• Classic statistical analysis, solely based on p-values, can be misleading.
• Misleading conclusions (false-positives and false-negatives) are frequent when hypotheses are unlikely, baseline rates are low, or observed differences are small.
• These errors persist even in large studies when relying only on p-values.

Conclusions:

• Bayes' theorem provides a more relevant posterior probability for hypothesis assessment in clinical trials.
• Adopting Bayesian conventions in the analysis and reporting of clinical trials can minimize errors.
• Revised statistical policies are needed to overcome the limitations of classic statistical theory.