Confidence intervals for an effect size measure based on the Mann-Whitney statistic. Part 1: general issues and...

Area of Science:

• Statistics
• Biostatistics
• Machine Learning

Background:

• Comparing distributions of random variables is crucial in various scientific fields.
• Existing effect size measures may not fully capture the separation between distributions.

Purpose of the Study:

• To propose a general measure of effect size, theta, for quantifying distribution separation.
• To introduce an estimator for theta based on the Mann-Whitney U statistic.
• To explore the impact of data discretization and develop confidence interval methods.

Main Methods:

• Defined theta as Pr[Y > X] + (1/2)Pr[Y = X].
• Estimated theta using U/mn, a generalization of the Mann-Whitney U statistic.
• Investigated discretization effects and developed tail-area-based confidence intervals.

Main Results:

• Theta provides a comprehensive measure of distribution separation.
• The U/mn estimator is equivalent to the area under the receiver operating characteristic curve.
• Developed methods applicable to small samples and extreme outcomes.

Conclusions:

• Theta is a versatile and interpretable effect size measure.
• The proposed estimation and inference methods offer practical solutions for comparing distributions.
• The approach is robust to data discretization and suitable for various sample sizes.