Quality of clinical practice guidelines in Japan: a time trend analysis using AGREE II scores

Previous studies evaluating the quality of clinical practice guidelines (CPGs) using the Appraisal of Guidelines for Research & Evaluation (AGREE) II tool mainly focus on domain scores. The current study analyses item-level AGREE II scores over time to identify trends in CPG quality and key areas for improvement.

We included all Japanese CPGs developed using evidence-based medicine (EBM) methods and published between 2011 and 2022. Each CPG was evaluated by three independent appraisers using the AGREE II, with item scores ranging from 1 (strongly disagree) to 7 (strongly agree). Time trends in item scores were analysed using Spearman's rank correlation coefficients. The items were grouped based on their 2011 scores and whether improving trends were observed.

A total of 604 CPGs were evaluated, with the mean item score in 2011 of 4.96. No item had both a high score and an improving trend. A total of 13 items had high scores but no improving trend, while six low-scoring items exhibited improvement. A total of four items (5, 9, 14, 21) had low scores and no improving trend, with items 14 and 21 displaying worsening trends.

Items 14 (Updating Procedure) and 21 (Monitoring/Auditing Criteria) were identified as key areas requiring improvement. Addressing post-publication efforts and disseminating best practices may improve the overall CPG quality.