Diabetes quality assurance checklist: assessment of inter-rater and intra-rater reliability

The Diabetes Quality Assurance (DQA) Checklist was developed to measure adherence to standards of diabetes care. Two raters simultaneously scored a convenience sample of 23 charts of patients with diabetes. These raters scored each chart again 5-7 weeks later. Data obtained were used to assess inter-rater and intra-rater reliability. Inter-rater reliability was estimated for basic assessment at Time 1 (r = 0.94, 95% CI 0.86-0.97) and at Time 2 (r = 0.91, 95% CI 0.81-0.96); and for high-risk assessment at Time 1 (r = 0.88, 95% CI 0.73-0.95). Intra-rater reliability for the basic assessment was estimated for Reviewer 1 (r = 0.84, 95% CI 0.65-0.93) and for Reviewer 2 (r = 0.75, 95% CI 0.49-0.89); the high-risk estimate for Review 1 was 0.60 (95% CI 0.25-0.81) and for Reviewer 2 0.97 (95% CI 0.94-0.99). The DQA Checklist is useful for monitoring and assessing diabetes care.