Validity of performance indicators for assessing prescribing quality: the case of asthma

The aim of this study was to assess the concurrent validity between the identification of sub-optimal treatment based on clinical information and computer generated indicators. Indicators that are associated with sub-optimal treatment in one of the four steps of asthma management were assessed.

The ability of each indicator to identify patients with sub-optimal asthma treatment from computerised general practitioner (GP) prescription records was assessed by comparing them with the results of an individual patient assessment using clinical data.

Chronic asthma patients ( n=146) registered with 16 Dutch GPs.

The sensitivity and positive predictive value (PPV) of each performance indicator was determined.

The step-1 indicator, focusing on patients not prescribed a short-acting beta-agonist, had an acceptable sensitivity (0.86), but a low PPV (0.52). The two step-2 indicators, targeting under-prescription of inhaled corticosteroids, had sensitivities of 0.74 and 0.37 and PPVs of 0.46 and 0.71, respectively. The step-3 indicator, which targeted under-dosing of inhaled corticosteroids, had a sensitivity of 0.07 and a PPV of 0.2. The fourth indicator, focusing on under-prescription of long-acting beta-agonists, could not be validated due to inadequate numbers of patients with severe asthma in our study sample.

None of the indicators investigated was considered valid for assessing prescriber performance, despite having good face and content validity. Performance indicators that have not been validated can only provide a broad-brush approach for assessing prescribing quality and should be used with extreme caution.