The role of pilot studies in the economic evaluation of health technologies

An increasing number of economic evaluations are being conducted alongside clinical trials. While this practice offers the prospect of collecting comprehensive and accurate cost data, it requires considerable time and effort. In the case of clinical data, key analytic decisions such as which data to collect and sample size are often made with reference to smaller (pilot) trials. However, this approach is not normally followed in the case of economic evaluation. This study was based on a recently completed health technology assessment comparing conventional radiotherapy with continuous hyperfractionated accelerated radiotherapy (CHART) for patients with head and neck cancer or carcinoma of the bronchus. In the full health technology assessment, cost data were available for 526 head and neck patients (314 CHART and 212 conventional therapy) and 286 bronchus patients (175 CHART and 109 conventional therapy). In order to simulate a pilot study, data were extracted for the patients recruited to both trials in the first 3 months. These were then compared with the full data set in order to assess whether such a pilot study would have given useful guidance on: a) the usefulness of undertaking a full study; b) the sample size required; and c) the important resource items for which comprehensive data collection would be required. Pilot studies can be helpful in determining the likely advantages of undertaking full economic evaluations and in identifying important resource items. Therefore, it is important that clinical researchers and research funding bodies create the necessary time window to enable such studies to take place. However, formal sample size calculations are more difficult to perform on limited data, since they also require knowledge of the unit cost (or prices) to be attached to the resource items and the correlation between costs and clinical effects.