A test for reporting bias in trial networks: simulation and case studies

Area of Science:

• Clinical trial methodology
• Biostatistics
• Evidence synthesis

Background:

• Networks of clinical trials are increasingly used to compare multiple treatments for the same condition.
• Randomized trial evidence may be incomplete due to reporting biases.
• A novel statistical test is proposed to identify reporting bias within trial networks.

Purpose of the Study:

• To develop and evaluate a statistical test for detecting reporting bias in networks of clinical trials.
• To assess the performance of this test in simulations under various selective reporting scenarios.
• To apply the test to real-world examples of antidepressant and antipsychotic trial networks.

Main Methods:

• The proposed test compares the observed number of statistically significant trial results against the expected number within a network.
• Simulation studies were conducted to evaluate the test's type I error rate and statistical power.
• The test was applied to published data and Food and Drug Administration (FDA) data for antidepressant and antipsychotic trials.

Main Results:

• The test demonstrated a maintained type I error rate and moderate power in simulations, particularly when between-trial variance was not substantial.
• A positive test result moderately to markedly increased the likelihood of reporting bias, while a negative result was less informative.
• Application to trial networks indicated an excess of significant results in published data but not in FDA data, suggesting bias.

Conclusions:

• The developed test can effectively signal an excess of statistically significant findings in trial networks.
• This provides evidence for potential publication bias or other selective reporting and outcome biases.
• The test serves as a valuable tool for improving the reliability of evidence synthesis from trial networks.