Significance Testing for Differences Between Baseline Variables Versus the I2 Test in Detecting Selection Bias in...

Area of Science:

• Biostatistics
• Clinical Trial Methodology

Background:

• Selection bias is a critical threat to the validity of randomized controlled trials (RCTs).
• Accurate detection of selection bias is essential for reliable clinical trial interpretation.

Purpose of the Study:

• To compare the effectiveness of the I² test versus traditional p-value based significance tests in detecting selection bias in RCTs.
• To test the null hypothesis that sensitivities and specificities of these tests do not differ significantly.

Main Methods:

• Simulated 100 RCTs (50 biased, 50 unbiased) with 100 subjects per group using MS Excel.
• Applied both p-value significance tests and the I² test to all simulated trials.
• Calculated sensitivities, specificities, and compared them statistically using z-tests.

Main Results:

• Both tests demonstrated 100% specificity, with no false positives.
• The I² test achieved a sensitivity of 76.00%, significantly outperforming the p-value test's sensitivity of 24.00%.
• Statistical analysis confirmed the superiority of the I² test (z = 5.2; p < 0.0001).

Conclusions:

• The I² test is a more effective tool for identifying selection bias in RCTs compared to p-value based significance tests.
• The findings reject the null hypothesis, indicating a significant difference in the sensitivity of the two methods.