Trial-Adjusted Versus Generic Simulated Comparator Trial (SCT) Settings for Selection Bias Appraisal Using the I2...

Area of Science:

• Biostatistics
• Clinical Trial Methodology
• Meta-Analysis

Background:

• Selection bias is a critical concern in clinical trials, potentially affecting study validity.
• The I² statistic is commonly used to assess heterogeneity, but its application in detecting selection bias requires careful parameterization.
• Simulated Comparator Trial (SCT) settings influence the performance of the I² test.

Purpose of the Study:

• To evaluate if trial-adjusted Simulated Comparator Trial (SCT) settings enhance the detection of selection bias compared to generic SCT settings.
• To determine if trial-adjusted SCT settings affect the detection of selection bias differently in smaller versus larger clinical trials.

Main Methods:

• Retrospective analysis of 67 randomized controlled trials previously assessed for selection bias.
• Re-testing trials using I² with both generic and trial-adjusted SCT settings.
• Systematic adjustment of SCT sample sizes (N_SCT = 200, 400, 800, 1200) to assess impact on I² estimates.

Main Results:

• Trial-adjusted SCT settings identified significantly more instances of selection bias (13/67) compared to generic settings (4/67), with an Odds Ratio of 3.79 (p=0.03).
• Increasing SCT sample size with trial-adjusted settings showed a trend towards more bias detection, but this was not statistically significant (OR: 1.30; p=0.53).
• The improved detection of bias using trial-adjusted settings was consistent across different trial sizes.

Conclusions:

• Trial-adjusted SCT settings demonstrably increase the odds of identifying selection bias in clinical trials.
• The enhanced sensitivity of trial-adjusted SCT settings did not significantly change the odds of detecting bias in smaller trials (n=100-199).
• This refined approach offers improved accuracy in bias assessment within meta-analyses.