Evaluation of the Quality of Japanese Clinical Practice Guidelines Using Appraisal of Guidelines for Research and...

Clinical Practice Guidelines (CPGs) play a crucial role in promoting evidence-based practice and reducing variations in healthcare delivery. In Japan, approximately 50 CPGs are published annually. However, few studies have systematically evaluated their methodological quality or examined how it has evolved over time using validated assessment tools.

This study aimed to evaluate trends in the methodological quality of Japanese CPGs published between 2011 and 2022 using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument.

We identified Japanese CPGs published between 2011 and 2022 using the Japanese guideline clearinghouse, jointly operated by the Toho University Medical Media Center and the Japan Medical Abstracts Society. This clearinghouse comprehensively collects all CPGs published in Japan. A total of 658 CPGs were identified by expert librarians based on predefined criteria. Three trained reviewers independently assessed the CPGs using the AGREE II, a validated tool that includes 23 items across 6 domains and one overall assessment item. CPGs intended primarily for patients, translated versions and those not based on evidence-based methodologies were excluded. Domain scores were calculated as percentages of the maximum possible scores. Time trends were evaluated using Spearman's rank correlation coefficient, and inflection points were assessed using joinpoint regression. Paired and independent t-tests were used to compare domain scores between first and revised editions, and between CPGs published in 2011-2014 and those published in 2015-2022.

Of the 604 CPGs evaluated, 248 were first editions and 356 were revised editions. Domain 1 (Scope and Purpose) had the highest average score (81.5%), followed by Domain 4 (Clarity of Presentation, 77.0%), while Domain 5 (Applicability) had the lowest (47.7%). Domain 6 (Editorial Independence) showed a significant upward trend (p = 0.01), whereas Domains 1 and 4 declined over time (p < 0.001). Inflection points were identified in 2015 for Domain 4 and in 2018 for overall quality scores. Revised editions showed significant improvements in Domain 5 (p < 0.001) and 6 (p = 0.005), while Domain 1 significantly declined (p = 0.001). CPGs revised and published in 2015-2022 showed fewer improvements than those revised and published in 2011-2014.

This study highlights both progress and persistent challenges in the quality of Japanese CPGs. Improvements in editorial independence suggest increased awareness of conflict of interest management.