Mapping and Quality Appraisal of Artificial Intelligence Preferential Reporting Checklists, Items, Guidelines, and...

The integration of artificial intelligence (AI) in healthcare has garnered significant scholarly attention, particularly in areas such as medical image analysis, prognosis, and treatment. Despite its potential, concerns regarding AI's reliability and application persist, prompting the development of guidelines aimed at standardizing its use in medicine. This study aims to evaluate the current content and quality of AI guidelines in healthcare, focusing on identifying gaps and providing a critical appraisal of existing checklists.

Comprehensive bibliometric analysis, Altmetric analysis, and systematic review were conducted, utilizing the AGREE II Tool for quality appraisal. The systematic search spanned Scopus, PubMed, and Dimension AI databases, focusing on English-language, open-access articles related to AI reporting guidelines. Two reviewers independently evaluated the data, with manual extraction performed in Microsoft Excel. The AGREE II Tool assessed six domains of guideline quality.

The search yielded 2477 articles, ultimately identifying 27 AI-specific reporting guidelines published between 2020 and 2025. The analysis revealed significant variations in quality across the AGREE II domains. Among the very first and most impactful were SPIRIT-AI, CONSORT-AI, MINIMAR, and CLAIM, all prioritized structured reporting but were hampered by their timing-based when there were few AI trials-resulting in limited applicability and risk of missing older AI terminologies. However, STAR-machine learning (ML), APPRAISE-AI, and CLEAR exhibited more general, domain-specific frameworks, whereas checklists such as CHEERS-AI, CREMLS, and MI-CLEAR-large language model (LLM) showcased limited author diversity in contribution. However, many guidelines exhibited weaknesses in methodological rigor and stakeholder involvement, limiting their practical applicability.

The findings emphasize the need for evidence-based updates to AI reporting guidelines to ensure methodological integrity amid rapid advancements. Increased expert involvement and stakeholder engagement are crucial for enhancing the guidelines' applicability and rigor, addressing AI complexities in health research, and adapting reporting frameworks to evolving AI technologies in healthcare.