Are estimands being correctly used? A review of UK research protocols

The use of estimands in clinical trials was formalised with the adoption of the final International Conference on Harmonisation E9 Addendum on Estimands and Sensitivity Analysis in Clinical Trials (ICH E9(R1) Addendum) in November 2019. The declared objective of the ICH E9(R1) Addendum is to bring clarity and transparency to the research question of interest. For this to be achieved, the estimand must be described in accordance with the requirements of the ICH E9(R1) Addendum so that the target treatment effect is clear to all stakeholders. Previous reviews of publications and published protocols have found that few trials explicitly defined the primary estimand. To obtain a more complete picture of how the use of estimands has changed over time, whether trials are using estimands correctly (i.e. correctly defining the five attributes of an estimand), and which strategies are being used to handle intercurrent events, we obtained access to an extensive database of original research protocols (n = 29,212) submitted to the United Kingdom's Health Research Authority, which oversees ethical review of clinical trials.

Protocols were eligible for review if they included the term 'estimand' and attempted to define at least one attribute of the primary estimand. For eligible protocols, we extracted information on trial characteristics such as whether the trial was randomized and the therapeutic area, as well as whether the estimand attributes used for the primary outcome were correctly defined, and which strategies were used to handle intercurrent events.

We found that the number of protocols defining a primary estimand increased starkly with publication of the ICH E9(R1) Addendum (approximately 3 protocols/year pre-ICH E9(R1) Addendum vs. 18 protocols / year during the consultation period vs. 23 protocols in the year following the adoption of the ICH E9(R1) Addendum). However, the description of the primary estimand was suboptimal; many protocols failed to mention specific attributes (such as population or treatment conditions) in the estimand description, and many protocols incorrectly defined estimand attributes (e.g. by describing the estimand population based on their analysis population).

Although release of the ICH E9(R1) Addendum has dramatically increased the use of estimands in clinical trials, their reporting is suboptimal. There is still work to be done to ensure estimands reach their full potential in bringing clarity and focus to research questions.