A cautionary note on applying scores in stratified data

When rank tests are used to analyze stratified data, three methods for assigning scores to the observations have been proposed: (S) independently within each stratum (see Lehmann, 1975, Nonparametrics: Statistical Methods Based on Ranks; San Francisco: Holden-Day); (A) after aligning the observations within each stratum and then pooling the aligned observations (Hodges and Lehmann, 1962, Annals of Mathematical Statistics 33, 482-497); and (P) after pooling the observations across all strata (that is, without alignment) (Mantel, 1963, Journal of the American Statistical Association 58, 690-700; Mantel and Ciminera, 1979, Cancer Research 39, 4308-4315). Test statistics are formed for each method by combining the stratum-specific linear rank tests using the assigned scores. We show that method P is sensitive to the score function used in the case of two moderately sized strata. In general, we recommend methods S and A for use with moderate to large-sized strata.