Cash transfers and HIV/HSV-2 prevalence: A replication of a cluster randomized trial in Malawi

In this paper we perform a replication analysis of "Effect of a cash transfer programme for schooling on prevalence of HIV and herpes simplex type 2 in Malawi: a cluster randomised trial" by Sarah Baird and others published in "The Lancet" in 2012. The original study was a two-year cluster randomized intervention trial of never married girls aged 13-22 in Malawi. Enumeration areas were randomized to either an intervention involving cash transfer (conditional or unconditional of school enrollment) or control. The study included 1708 Malawian girls, who were enrolled at baseline and had biological testing for HIV and herpes simplex virus type 2 (HSV-2) at 18 months. The original findings showed that in the cohort of girls enrolled in school at baseline, the intervention had an effect on school enrollment, sexual outcomes, and HIV and HSV-2 prevalence. However, in the baseline school dropout cohort, the original study showed no intervention effect on HIV and HSV-2 prevalence.

We performed a replication of the study to investigate the consistency and robustness of key results reported. A pre-specified replication plan was approved and published online. Cleaned data was obtained from the original authors. A pure replication was conducted by reading the methods section and reproducing the results and tables found in the original paper. Robustness of the results were examined with alternative analysis methods in a measurement and estimation analysis (MEA) approach. A theory of change analysis was performed testing a causal pathway, the effect of intervention on HIV awareness, and whether the intervention effect depended on the wealth of the individual.

The pure replication found that other than a few minor discrepancies, the original study was well replicated. However, the randomization and sampling weights could not be verified due to the lack of access to raw data and a detailed sample selection plan. Therefore, we are unable to determine how sampling influenced the results, which could be highly dependent on the sample. In MEA it was found that the intervention effect on HIV prevalence in the baseline schoolgirls cohort was somewhat sensitive to model choice, with a non-significant intervention effect for HIV depending on the statistical model used. The intervention effect on HSV-2 prevalence was more robust in terms of statistical significance, however, the odds ratios and confidence intervals differed from the original result by more than 10%. A theory of change analysis showed no effect of intervention on HIV awareness. In a causal pathway analysis, several variables were partial mediators, or potential mediators, indicating that the intervention could be working through its effect on school enrollment or selected sexual behaviors.

The effect of intervention on HIV prevalence in the baseline schoolgirls was sensitive to the model choice; however, HSV-2 prevalence results were confirmed. We recommend that the results from the original published analysis indicating the impact of cash transfers on HIV prevalence be treated with caution.