Comparison of profile-likelihood-based confidence intervals with other rank-based methods for the two-sample problem...

Area of Science:

• Statistics
• Clinical Trials
• Biostatistics

Background:

• The relative effect is a key effect size measure for ordered categorical data in randomized clinical trials.
• The Wilcoxon-Mann-Whitney (WmW) test's null hypothesis is identical distributions, not just a 50% relative effect.
• The Brunner-Munzel (BM) test's null hypothesis is a 50% relative effect, suitable for various data types.

Purpose of the Study:

• To compare the actual type I error rates of profile-likelihood confidence intervals for relative effect with rank-based methods.
• To evaluate performance under various conditions, including unequal distributions and sample sizes.

Main Methods:

• Simulation studies were conducted to assess type I error rates at a 50% relative effect.
• Profile-likelihood-based confidence intervals for relative effect were compared with the WMW and BM tests.
• No distribution assumptions were required for the profile-likelihood method or the BM test.

Main Results:

• Profile-likelihood and BM tests showed type I error rates close to nominal levels in large/medium samples, even with unequal distributions.
• The WMW test exhibited significant deviations from nominal levels under unequal distributions and sample sizes.
• In small samples, BM test rates were slightly above nominal, and profile-likelihood rates were higher.
• A paradoxical example demonstrated WMW's sensitivity to allocation ratios, unlike profile-likelihood and BM tests.

Conclusions:

• Profile-likelihood and BM tests are more reliable for assessing relative effect size in clinical trials than the WMW test, particularly with distributional or sample size imbalances.
• The WMW test's type I error rates can be unpredictable based on sample allocation ratios.