Related Experiment Videos
Inter- and intrarater reliability of retrospective drug utilization reviewers
I H Zuckerman1, D M Mulhearn, C J Metge
1School of Pharmacy, University of Maryland, Baltimore 21201, USA. zuckerma@pharmacy.ab.umd.edu
Objective:
To assess inter- and intrarater reliability among 23 pharmacist and physician retrospective drug utilization reviewers and to assess interrater reliability after a reviewer training session.
Design:
Exploratory study.
Setting:
Maryland Medicaid's retrospective drug utilization review (DUR) program.
Participants:
23 physician and pharmacist retrospective drug utilization reviewers.
Interventions:
None.
Main Outcome Measures:
Profiles rated as "intervention indicated" or "intervention not indicated." Cochran's Q test, overall percent agreement, and the unweighted kappa statistic were used in the analysis of review consistency.
Results:
Intrarater reliability showed substantial consistency among the 23 reviewers; the percent agreement was 82.9% with kappa = 0.66. Interrater reliability, however, was poor, with an overall agreement of 69.6% and kappa = 0.16. Interrater reliability was also poor after a one-hour reviewer training session (agreement 81.8%, kappa = -0.19).
Conclusion:
The implicit review process used in the retrospective DUR program that we evaluated was unreliable. Since reliability is a necessary but not sufficient condition for validity of an indicator of inappropriate drug use, the validity of the DUR implicit review process is in question.