A critical review of methodologies used in pharmaceutical pricing policy analyses

Robust evidence from health policy research has the potential to inform policy-making, but studies have suggested that methodological shortcomings are abundant. We aimed to identify common methodological weaknesses in pharmaceutical pricing policy analyses. A systematic review (SR) of studies examining pharmaceutical pricing policies served as basis for the present analysis. We selected all studies that were included in the SR (n = 56), and those that were excluded from the SR due to ineligible study designs only (n = 101). Risk of bias was assessed and specific study design issues were recorded to identify recurrent methodological issues. Sixty-one percent of studies with a study design eligible for the SR presented with a high risk of bias in at least one domain. Potential interference of co-interventions was a source of possible bias in 53% of interrupted time series studies. Failing to consider potential confounders was the primary cause for potential bias in difference-in-differences, regression, and panel data analyses. In 101 studies with a study design not eligible for the SR, 32% were uncontrolled before-after studies and 23% were studies without pre-intervention data. Some of the methodological issues encountered may be resolved during the design of a study. Awareness among researchers on methodological issues will help improve the rigor of health policy research in general.