Comparing Approaches for Estimating Counterfactual HIV Incidence Among Populations With High Vulnerability to HIV in...

The availability of highly efficacious HIV pre-exposure prophylaxis (PrEP) makes it unethical or infeasible to conduct inactive/placebo-controlled trials to evaluate new HIV prevention options. As a result, randomized active-control non-inferiority trials are typically employed; however, they require large sample sizes and extended follow-up. Moreover, their results can be hard to interpret. An alternative approach compares HIV incidence among individuals receiving a new PrEP product to a counterfactual incidence estimate-an estimate of what the HIV incidence would have been in the absence of PrEP.

We leveraged data from three studies (AMP, Sabes and ImPrEP seroincidence) conducted among men who have sex with men and transgender persons in Lima, Peru, between 2013 and 2022 to estimate the counterfactual HIV incidence for two target populations represented by a clinical trial (AMP) and a non-interventional cohort study (Sabes). We evaluated three estimation methods: prospective cohort follow-up, recency testing and rectal gonorrhoea (RG) approaches. These approaches were compared directly using data from the same study, and we further assessed population adjustment approaches by comparing estimates across studies.

All three methods produced consistent HIV incidence estimates when applied to data collected from the same study. ​However, estimates differed when data from external studies were used, even after propensity score (PS) adjustment. For example, estimates for the AMP population using Sabes or ImPrEP data remained higher than the AMP gold-standard estimate. In contrast, adjusted estimates for the Sabes population using AMP or ImPrEP data were lower than the Sabes follow-up estimate. These differences underscore the challenges of applying external data and highlight the role of unmeasured population heterogeneity.

Counterfactual HIV incidence estimates can support evaluation of new PrEP agents when direct placebo comparisons are not possible. Estimates from recency-testing and RG approaches are promising when prospective follow-up is infeasible, but their validity depends on data quality and population similarity. PS methods improve comparability across populations but cannot fully account for unobserved differences. Triangulating across methods and sources can improve confidence in counterfactual estimates, which should be interpreted with careful attention to population context.