Confidence intervals for the Mann-Whitney test.

Area of Science:

• Statistics
• Biostatistics
• Non-parametric statistics

Background:

• The Mann-Whitney test is a frequent non-parametric alternative to the two-sample t-test.
• Confidence intervals for effect sizes are rarely reported alongside the Mann-Whitney test.
• Existing effect size measures (median difference, location shift) do not directly align with the Mann-Whitney test statistic.

Purpose of the Study:

• To evaluate confidence interval construction for the Mann-Whitney test's effect size, specifically the probability of X < Y.
• To review and compare promising methods for confidence interval estimation.
• To address issues with variance estimators and small sample sizes impacting interval accuracy.

Main Methods:

• Focus on the probability of random variable X being lower than random variable Y (P(X
• Review and analyze various proposed methods for constructing confidence intervals for P(X
• Investigate properties of different variance estimators and address small sample size challenges.

Main Results:

• Identified scenarios where existing confidence interval approaches yield inadequate coverage probabilities.
• The DeLong variance estimator is recommended as a reliable option across various scenarios.
• Confidence intervals constructed on the logit scale demonstrate improved accuracy, avoiding boundary issues (0 or 1) and poor coverage.

Conclusions:

• The DeLong variance estimator is robust for Mann-Whitney confidence intervals.
• Utilizing the logit scale for confidence interval construction is crucial for accurate coverage and interpretation.
• A proposed correction enhances coverage probability, particularly in cases with complete sample separation.