Agreement between pharmacists for problem identification: an initial quality measurement of cognitive services

Recent changes in national reimbursement policies expand the ability of pharmacists to seek reimbursement for cognitive services. The quality of pharmacist-provided cognitive services has, until now, remained unassessed. Pharmacists should demonstrate the quality and value of their work to ensure the continued and expanded acceptance of reimbursement for their services. A preliminary step in assessing quality is to compare agreement between pharmacists for basic problem identification.

To quantify agreement between pharmacist reviewers for problem identification among Utah Medicaid recipients.

Five pharmacists retrospectively reviewed drug regimens, patient characteristics, diagnosis codes, and procedures for 80 Medicaid patients in September 2008 and identified drug-related problems (DRPs) in 15 predetermined categories. Data for each patient were reviewed twice, and each combination of 2 pharmacists reviewed the same 8 patients' information. We calculated a reliability coefficient to compare the number of DRPs identified and used prevalence and bias adjusted kappa (PABAK) to determine interrater reliability for the presence of a specific DRP.

Of the 15 DRPs categorized by pharmacist reviewers, 1 (untreated indications) had a PABAK coefficient of 0.20, indicating a relatively low level of agreement between reviewers. All other DRP categories had good to excellent agreement, with PABAK coefficients ranging between 0.43 and 0.98.

Pharmacist reviewers exhibited less variability in DRP identification or categorization than had been expected for most categories. This work supports the conclusion that pharmacists in our center provide a basic and necessary level of quality for problem assessment. Future work is needed to document the impact of this quality on patient outcomes.