Desirability of outcome ranking (DOOR) analysis for multivariate survival outcomes with application to ACTT-1 trial

Desirability of outcome ranking (DOOR) is a paradigm for the design, monitoring, analysis, interpretation, and reporting of clinical trials based on patient-centric benefit-risk evaluation, developed to address limitations of existing approaches and advance clinical trial science. The first step in implementing DOOR is defining an ordinal DOOR outcome representing a global patient-centric response, a cumulative summary of the benefits and harms for an individual patient. This article aims to develop an analysis methodology for the setting where the DOOR outcome is a progressive time-varying state, and there is interest in event times and times that patients spend in more and less desirable states.

We develop methods to estimate and make inferences about the temporal treatment effects. If the k-levels of the DOOR outcome are monotone, then k - 1 non-overlapping Kaplan-Meier survival curves can be estimated and plotted. The areas under the curves asymptotically follow a multivariate Gaussian distribution. We apply restricted mean survival time (RMST) concepts to the ordinal Kaplan-Meier curves and provide steps for estimating the covariance structure.

Simulation studies demonstrate that the proposed methods perform well in practical settings. We generate censoring time under a uniform distribution and event times under a multi-state structure. The proposed estimators have small biases, the 95% confidence intervals have correct coverage probabilities, and the proposed tests accurately control the type I error rate under the null hypothesis. We illustrate the methods using data from Adaptive COVID-19 Treatment Trial (ACTT-1), a clinical trial that compared remdesivir vs placebo for the treatment of COVID-19 infection.

Ordinal DOOR outcomes, which incorporate benefits and harms and represent an overall patient response, have recently been recommended by the Council for International Organizations of Medical Sciences (CIOMS) as a standard approach to benefit:risk analysis. Such endpoints recognize the cumulative nature of outcomes on patients, account for correlations between efficacy and safety, incorporate multivariate survival outcomes, offer generalizability to inform clinical practice, and recognize finer gradations of patient response and binary outcomes. Robust and interpretable analysis methodologies for ordinal outcomes are needed.

Restricted mean survival time is a useful nonparametric approach for robust treatment effect estimation. We provide a framework for inference using multiple RMSTs to analyze DOOR and other ordinal outcomes using an interpretable time metric.