Reverse Bayesian Implications of p-Values Reported in Critical Care Randomized Trials.

Area of Science:

• Biostatistics
• Clinical Trials
• Medical Research

Background:

• Null-hypothesis statistical testing (NHST) is widely used in medical research.
• Misinterpretations of p-values are frequent, potentially leading to erroneous conclusions.
• The implications of observed p-values in critical care randomized controlled trials (RCTs) require careful examination.

Purpose of the Study:

• To assess the implications of observed p-values in critical care RCTs.
• To determine the false positive rates and positive predictive values associated with different p-value thresholds.
• To evaluate the credibility and replicability of findings based on p-value significance.

Main Methods:

• Analysis of three cohorts of published RCTs: adult mortality, pediatric mortality, and recent consecutive trials with p-value ≤0.10.
• Inclusion of trials from six high-impact journals.
• Calculation of reverse Bayesian implications, reported as percentages with interquartile ranges.

Main Results:

• P-values ≤0.005 were observed in 5.1% of adult RCTs, 1.7% of pediatric RCTs, and 41.1% of consecutive RCTs.
• P-values between 0.05 and 0.0051 exhibited high false positive rates (realistic: 64.3% in adult RCTs) and low positive predictive values (realistic: 28.0% in adult RCTs).
• P-values ≤0.005 demonstrated lower false positive rates (realistic: 7.7% in adult RCTs) and higher positive predictive values, with replication probability approaching 90% only when p ≤0.005.

Conclusions:

• Findings with p-values between 0.05 and 0.0051 in critical care RCTs have questionable credibility due to high false positive rates.
• A p-value threshold of ≤0.005 is necessary to achieve a high positive predictive value (>80%) and a high probability of replication (>90%).
• The credibility of ruling out an effect with p-values >0.05 to 0.10 is low, necessitating cautious interpretation of statistical significance.