Is NICE infallible? A qualitative study of its assessment of treatments for attention-deficit/hyperactivity disorder...

Conclusions of the recent NICE technology appraisal of treatments for attention-deficit/hyperactivity disorder (ADHD) differ from recommendations by other Health Technology Assessment (HTA) agencies, such as the Scottish Medicines Consortium (CMS) and the Australian Pharmaceutical Benefits Advisory Committee (PBAC). NICE did not identify differences on grounds of clinical effectiveness between treatment options studied and issued technology guidance based on clinical profiles of compounds and on drug acquisition costs. The aim of the present study was to explore the robustness of NICE assessment methods when addressing a complex clinical problem such as the evaluation of ADHD treatment strategies. This robustness will be of interest to international policy-makers, given the widespread perception of NICE as a role model for the implementation of HTAs including economic evaluation.

A qualitative case study was performed to critically appraise the technology assessment report (AR) underlying NICE conclusions, including a systematic search for and analysis of relevant literature.

The AR produced on behalf of NICE was found to exhibit a range of anomalies. Search criteria were not applied consistently, and the available clinical evidence was not used optimally; selection of clinical endpoints and clinical trials for analysis were idiosyncratic. The primary cost-effectiveness model relied on six short-term studies only, and secondary extensions combined heterogeneous study designs and different clinical endpoints. Neither the distinction between efficacy and effectiveness nor the role of treatment compliance in ADHD was addressed adequately. Long-term extensions of the model were impaired by use of inappropriate discount rates and absence of consideration of long-term sequelae associated with ADHD.

A review of the literature strongly suggests that the NICE assessment of ADHD treatment strategies was incomplete and likely prone to bias. It is concluded that NICE did not adequately accommodate a complex clinical decision problem. Although the present qualitative case study of one assessment cannot, and was not designed to, invalidate the NICE approach to economic evaluation of healthcare programs, this observation may have potentially far-reaching implications for the generalizability of NICE-like approaches.