Algorithmic benchmark modulation: A novel method to develop success rates for clinical studies

High-quality decision-making in the pharmaceutical industry requires accurate assessments of the Probability of Technical Success of clinical trials. Failure to do so will lead to lost opportunities for both patients and investors. Pharmaceutical companies employ different methodologies to determine Probability of Technical Success values. Some companies use power and assurance calculations; others prefer to use industry benchmarks with or without the overlay of subjective modulations. At AstraZeneca, both assurance calculations and industry benchmarks are used, and both methods are combined with modulations.

AstraZeneca has recently implemented a simple algorithm that allows for modulation of a Probability of Technical Success value. The algorithm is based on a set of multiple-choice questions. These questions cover a comprehensive set of issues that have historically been considered by AstraZeneca when subjective modulations to Probability of Technical Success values were made but do so in a much more structured way.

A set of 57 phase 3 Probability of Technical Success assessments suggests that AstraZeneca's historical estimation of Probability of Technical Success has been reasonably accurate. A good correlation between the subjective modulation and the modulation algorithm was found. This latter observation, combined with the finding that historically AstraZeneca has been reasonably accurate in its estimation of Probability of Technical Success, gives confidence in the validity of the novel method.

Although it is too early to demonstrate whether the method has improved the accuracy of company's Probability of Technical Success assessments, we present our data and analysis here in the hope that it may assist the pharmaceutical industry in addressing this key challenge. This new methodology, developed for pivotal studies, enables AstraZeneca to develop more consistent Probability of Technical Success assessments with less effort and can be used to adjust benchmarks as well as assurance calculations.

The Probability of Technical Success modulation algorithm addresses several concerns generally associated with assurance calculations or benchmark without modulation: selection biases, situations where little relevant prior data are available and the difficulty to model many factors affecting study outcomes. As opposed to using industry benchmarks, the Probability of Technical Success modulation algorithm allows to accommodate project-specific considerations.