Closing the gap: A novel metric of change in performance

Interventions to improve performance of global programs in the HIV cascade of care are widespread and increasing the focus of implementation science. At present, however, there is no clear consensus on how to conceptualize their improvement at the program level. The commonly used measures of association, based on ratios of probabilities (or odds), have well-known defects in public health applications. They yield large effect sizes even when the absolute effects, and therefore the public health impact, are small. On the other hand, risk differences create problems because settings with higher baseline values are penalized. We aim to examine ways of quantifying improvement in each health center of a cluster-randomized trial in Uganda to accelerate antiretroviral therapy initiation among HIV-infected adults.

We formalize the concept of the 'improvement index,' defined as the fraction of gaps closed as a metric of improvement, and suggest that it has unique features and strengths when compared to risk ratios and risk differences.

Overall agreement between the different indices was not high, especially among health centers that were among the top 5 or 10. However, all ranking showed broad similarities at the far ends of the spectrum. On scatter plots, there was a positive linear relationship between the metrics, and the Bland Altman (B-A) plots were in agreement.

The improvement index can be used as an alternative measure of association in implementation science interventions. It can be useful for public health purposes as it demonstrates how much can be covered from the baseline.