Blinded subjective rankings as a method of assessing treatment effect: a large sample example from the Systolic...

Because many randomized clinical trials study more than one important outcome variable, evaluation of efficacy is often difficult and not completely satisfactory. This paper considers the use of a procedure for endpoint determination described by Follmann et al., that allows raters to integrate subjectively all relevant information about an individual's clinical course into a single univariate assessment. To explore the method's feasibility, we tested the procedure with data from a completed clinical trial, the Systolic Hypertension in the Elderly Program (SHEP). We provided raters blinded to treatment assignment with cards that schematically represent the clinical trajectories of SHEP study participants. The raters independently ranked these trajectories. The method combined ranks across raters to determine a single rank for each study participant; we used a rank procedure to test treatment effect. The major findings were: (i) the raters showed a high level of concordance of rankings; (ii) tests of treatment effect were highly statistically significant; (iii) three statistical methods were effective for implementing the ranking in the large study size case. These methods were use of: (a) scoring rules; (b) incomplete block designs, and (c) categorical ranking.