Causal estimands and confidence intervals associated with Wilcoxon-Mann-Whitney tests in randomized experiments

Area of Science:

• Statistics
• Biostatistics
• Epidemiology

Background:

• The P value from the Wilcoxon-Mann-Whitney test is frequently used in randomized experiments but often lacks causal effect estimates and confidence intervals.
• The Mann-Whitney parameter (ϕ) is the natural parameter for this test, representing the probability of a higher response in the treatment arm compared to the control arm, adjusted for ties.
• A related causal effect, ψ, represents the probability of a higher individual response under treatment versus control, adjusted for ties, but is nonidentifiable in randomized experiments.

Purpose of the Study:

• To frame the Mann-Whitney parameter (ϕ) as a causal parameter and distinguish it from the nonidentifiable causal effect ψ.
• To review Hand's paradox, where ψ may suggest treatment is worse while ϕ indicates the opposite.
• To explore nonparametric assumptions and bootstrap methods for bounding ψ and addressing Hand's paradox.

Main Methods:

• Causal inference framework applied to the Mann-Whitney parameter.
• Analysis of Hand's paradox and its implications for interpreting treatment effects.
• Utilizing nonparametric assumptions and bootstrap methods to estimate bounds on the causal parameter ψ.
• Investigating the role of the proportional odds parameter in Hand's paradox.

Main Results:

• The Mann-Whitney parameter (ϕ) can be interpreted causally but is distinct from the nonidentifiable parameter ψ.
• Hand's paradox highlights potential discrepancies between ϕ and ψ, where treatment may appear beneficial by ϕ but detrimental to most individuals by ψ.
• The paradox can occur for proportional odds parameters between 1/9 and 9, necessitating large effect sizes for consistent interpretation.
• Bootstrap methods provide a way to infer bounds on ψ.

Conclusions:

• The Mann-Whitney parameter (ϕ) offers a potentially useful, identifiable measure in randomized trials, but its interpretation requires careful consideration of the nonidentifiable parameter ψ and Hand's paradox.
• Large observed effects are crucial to ensure that a positive result for ϕ reliably indicates improved outcomes for the majority of individuals.
• The study underscores the importance of distinguishing between different causal effect measures and understanding their identifiability in randomized experiments, as demonstrated in a vaccine trial.