Related Experiment Video
Updated: May 26, 2026

A Two-interval Forced-choice Task for Multisensory Comparisons
Published on: November 9, 2018
The I2 Statistic As Selection Bias Test: Updated Threshold Limits
Steffen Mickenautsch1,2, Veerasamy Yengopal1
1Faculty of Dentistry, University of the Western Cape, Cape Town, ZAF.
Aim:
This study aims to revise the pre-specified I2 point estimate threshold limits of the trial-adjusted, simulated comparator trial (SCT)-based I2 test for selection bias in single randomised controlled trials (RCTs), in order to increase the percentage of testable RCTs. A further objective was to test the two null hypotheses: that, based on the test's revised I2 point estimate thresholds, the magnitude of trial effect estimates is not significantly positively correlated with the selection bias levels (H01), and that it does not differ significantly between RCTs with identified 'low' and 'high' selection bias (H02).
Methods:
Revision of the I2 point estimate threshold limits was based on multiple RCT simulation and the re-testing of 332 real-world RCTs for selection bias. The selection bias level (B%) of each RCT, using the revised limits, was determined. H01 was tested using Spearman's rank correlation, and H02 using the 2-tailed, independent samples t-test. The mean difference (MD) with 95% confidence interval (CI) between the mean absolute risk difference (RD) values with SD of RCTs with identified 'low' and 'high' selection bias was computed.
Results:
All 332 RCTs were testable, and none of the computed I2 point estimates fell outside the revised thresholds limits, increasing the proportion of testable RCTs from 71% to 100%. There was a statistically significant, positive, small correlation (0.1 ≤ |r| < 0.3) between the absolute RD values and established B% levels (Spearman's rho = 0.268, p < 0.0001). The mean absolute RD for RCTs with identified 'high' selection bias was 0.18 (SD = 0.16), compared with 0.10 (SD = 0.13) for RCTs with 'low' selection bias. The difference was statistically significant: (t = -4.65; p < 0.0001; MD 0.08; 95%CI: 0.05 - 0.11). The effect size (Cohen's d = 0.52) indicates a medium effect. Both null hypotheses were rejected.
Conclusion:
The revised threshold limits increased the utility of the trial-adjusted, SCT based I2 test from 71 to 100% testable RCTs. The rejection of both null hypotheses indicates that the previously established statistically significant, positive relationship between selection bias levels (B%) and effect estimates (absolute RD) is independent of the changes in the I² threshold limits.
Related Concept Videos
Accuracy and Errors in Hypothesis Testing
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
Critical Region, Critical Values and Significance Level
In hypothesis testing, a sample statistic is converted to a test statistic using z, t, or chi-square distribution. A critical region is an area under the curve in probability distributions demarcated by the critical value. When the test statistic falls in this region, it suggests that the null hypothesis must be rejected. As this region contains all those values of the test...
Bonferroni Test
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
Detection of Gross Error: The Q Test
Significance Testing: Overview
Behrens–Fisher Test
This test is...

