Using economic evaluations to make formulary coverage decisions. So much for guidelines

It is mandatory for drug manufacturers requesting formulary inclusion under the British Columbia (BC) provincial drug plan to submit a pharmacoeconomic analysis according to published guidelines. These submissions are reviewed by the Pharmacoeconomic Initiative (PI) of BC.

To assess the compliance of submitted studies with specific criteria outlined in the guidelines, to assess the methodological quality of individual submissions, and to demonstrate the importance of submitting guidelines-compliant pharmacoeconomic analyses.

All submissions between January 1996 and April 1999 assessed by the PI of BC were included. Submissions were reviewed according to a checklist to establish compliance with respect to choice of comparator drug, study perspective, sensitivity analysis, analytical horizon and discounting. Submissions were examined for association between analytical technique and author, and between source of submission and compliance. Association between compliance and recommendation for approval was also examined.

95 applications were reviewed. Seven submitted no analyses. There were 25 cost-comparison/consequence, 14 cost-effectiveness, 11 cost-minimisation, 9 cost-utility/benefit and 29 budget-impact analyses. 65 of these 88 submissions failed to comply with guidelines. Of these, 45% used an inappropriate comparator drug, 61% lacked a sensitivity analysis, 73% used a third-party payer and excluded a societal perspective, 66% did not provide a long term evaluation and 25% did not specify any time horizon. 80% of noncompliant studies were cost-comparison/consequence or budget-impact analyses (p < 0.001, Fisher's Exact). Of 25 cost-comparison/consequence and 29 budget-impact analyses, 19 (76%) and 24 (83%), respectively, were industry-conducted, whereas cost-effectiveness (11 of 14) and cost-utility/benefit (6 of 9) analyses were mostly subcontracted to private consultants or academics (p < 0.001, Fisher's Exact). 74% of all submissions (compliant and noncompliant) were not recommended by the PI for listing as a provincial drug plan benefit, 16% received approval for restricted benefit and 9% were recommended as full benefit. 80% of the noncompliant submissions were not recommended (p = 0.06, Fisher's Exact test). Moreover, a strong association between type of analysis and type of recommendation was found (p = 0.03, Fisher's Exact test). Cost-comparison/consequence and budget-impact analyses were less likely to be recommended. IMPLICATIONS OF FINDINGS: Our findings show poor compliance with guidelines, especially among industry-conducted studies. Possible explanations are lack of expertise in pharmacoeconomics and/or scepticism regarding the importance of guidelines and submission quality in decision making. As corroborated by the strong associations between type of recommendation and compliance, and between type of recommendation and type of analysis, these 2 characteristics have a significant impact on decision making.