Confidence intervals for the difference between independent binomial proportions: comparison using a graphical...

Area of Science:

• Biostatistics
• Statistical Inference

Background:

• Accurate confidence intervals for the difference between two independent binomial proportions are crucial for statistical analysis.
• Existing methods often exhibit poor coverage properties, particularly for one-sided intervals, potentially inflating Type I error rates.

Purpose of the Study:

• To graphically compare the coverage properties of various confidence interval methods for the difference between two independent binomial proportions.
• To evaluate both small-sample and large-sample properties, including two-sided and one-sided coverage.

Main Methods:

• Graphical analysis of coverage probabilities.
• Investigation of asymptotic methods for confidence interval calculation.
• Comparison of score-based methods, Brown-Li 'Jeffreys' method, Gart-Nam 'skewness-corrected' method, and a new hybrid method.

Main Results:

• Score-based methods generally provide the best two-sided coverage, with minor issues at lower confidence levels.
• The Brown-Li 'Jeffreys' method offers reasonable performance for hand calculation and better one-sided coverage than alternatives.
• Many asymptotic methods demonstrate poor one-sided coverage, leading to inflated Type I error rates when used for non-inferiority testing.
• The Gart-Nam method and a novel hybrid method show strong performance, with the Gart-Nam method offering bias and continuity corrections.

Conclusions:

• Score-based methods are recommended for reliable two-sided confidence intervals for binomial proportion differences.
• The Gart-Nam method, with its modifications, and the new hybrid method are strong candidates for accurate one-sided interval calculations.
• Researchers should carefully consider the one-sided coverage properties of chosen methods, especially for non-inferiority testing.