Development and preliminary verification of the evaluation system for clinical practice guidelines in China

Clinical practice guidelines can improve healthcare processes and patient outcomes; however, the quality of these guidelines varies greatly in China. The aim of this study was to construct a comprehensive instrument for the appraisal of clinical practice guidelines in China (AGREE-CHINA), and to validate its reliability as a tool for helping potential guideline users in assessing guideline quality.

First, an interdisciplinary working group was established for developing the methods. They also created a checklist as a tool according to the Appraisal of Guidelines, Research and Evaluation II (AGREE II) standards, considering the particularity of Chinese clinical practice. Next, the first draft of AGREE-China was developed by vote, modification, preliminary trial, and cross-verification. To ensure the objectivity, credibility, and reproducibility of the draft assessment, all of the checklists and standards were cross-reviewed fairly widely. Finally, AGREE-CHINA and AGREE II were used to assess the Chinese guidelines published in the past five years, and the results were compared.

The presented AGREE-CHINA covered five main checkpoints (science and rigor, effectiveness and safety, economy, usability and feasibility, and conflicts of interest) with each point divided into several more specific checkpoints. Definitions and rationales for each main checkpoint appear in the Appendix. The quality ratings based on the total scores of AGREE-China and AGREE II were consistent (r = 0.508, P = 0.020). Compared with AGREE II, the study showed a higher level of interrater-reliability for AGREE-CHINA overall (ICC = 0.957, P < 0.001). The mean time required for AGREE-CHINA was less than that for AGREE II; this was approximately 30 minutes for every assessment. User satisfaction was generally high.

This paper has presented the first edition of the AGREE-CHINA appraisal tool for clinical guidelines. It is quick and easy to use; it assesses and performs well in comparison to AGREE II. This first version of AGREE-CHINA will require further development and validation.