Methodologies for pragmatic and efficient assessment of benefits and harms: Application to the SOCRATES trial

Standard approaches to trial design and analyses can be inefficient and non-pragmatic. Failure to consider a range of outcomes impedes evidence-based interpretation and reduces power. Traditional approaches synthesizing information obtained from separate analysis of each outcome fail to incorporate associations between outcomes and recognize the cumulative nature of outcomes in individual patients, suffer from competing risk complexities during interpretation, and since efficacy and safety analyses are often conducted on different populations, generalizability is unclear. Pragmatic and efficient approaches to trial design and analyses are needed.

Approaches providing a pragmatic assessment of benefits and harms of interventions, summarizing outcomes experienced by patients, and providing sample size efficiencies are described. Ordinal outcomes recognize finer gradations of patient responses. Desirability of outcome ranking is an ordinal outcome combining benefits and harms within patients. Analysis of desirability of outcome ranking can be based on rank-based methodologies including the desirability of outcome ranking probability, the win ratio, and the proportion in favor of treatment. Partial credit analyses, involving grading the levels of the desirability of outcome ranking outcome similar to an academic test, provides an alternative approach. The methodologies are demonstrated using the acute stroke or transient ischemic attack treated with aspirin or ticagrelor and patient outcomes study (SOCRATES; NCT01994720), a randomized clinical trial.

Two 5-level ordinal outcomes were developed for SOCRATES. The first was based on a modified Rankin scale. The odds ratio is 0.86 (95% confidence interval = 0.75, 0.99; p = 0.04) indicating that the odds of worse stroke categorization for a trial participant assigned to ticagrelor is 0.86 times that of a trial participant assigned to aspirin. The 5-level desirability of outcome ranking outcome incorporated and prioritized survival; the number of strokes, myocardial infarction, and major bleeding events; and whether a stroke event was disabling. The desirability of outcome ranking probability and win ratio are 0.504 (95% confidence interval = 0.499, 0.508; p = 0.10) and 1.11 (95% confidence interval = 0.98, 1.26; p = 0.10), respectively, implying that the probability of a more desirable result with ticagrelor is 50.4% and that a more desirable result occurs 1.11 times more frequently on ticagrelor versus aspirin.

Ordinal outcomes can improve efficiency through required pre-specification, careful construction, and analyses. Greater pragmatism can be obtained by composing outcomes within patients. Desirability of outcome ranking provides a global assessment of the benefits and harms that more closely reflect the experience of patients. The desirability of outcome ranking probability, the proportion in favor of treatment, the win ratio, and partial credit can more optimally inform patient treatment, enhance the understanding of the totality of intervention effects on patients, and potentially provide efficiencies over standard analyses. The methods provide the infrastructure for incorporating patient values and estimating personalized effects.