Adherence of CLAIM in Artificial Intelligence Research: A Cross-Sectional Study

Area of Science:

• Medical Imaging AI Research
• Scientific Reporting Standards
• Artificial Intelligence in Medicine

Background:

• Artificial intelligence (AI) is increasingly used in medical imaging, necessitating standardized reporting to ensure research quality and reproducibility.
• The Checklist for Artificial Intelligence (AI) in Medical Imaging (CLAIM) was developed to guide the reporting of AI studies in this field.
• Assessing the adherence to CLAIM in high-impact medical imaging journals is crucial for understanding current reporting practices.

Purpose of the Study:

• To evaluate the extent to which AI research published in leading medical imaging journals adheres to the CLAIM guidelines.
• To identify factors influencing the reporting quality and compliance rate with CLAIM.
• To analyze temporal trends in CLAIM adherence over time.

Main Methods:

• A systematic search of AI research articles was conducted in prominent medical imaging journals indexed in the Web of Science Core Collection.
• Reporting scores and compliance rates were calculated based on the 42 items of the CLAIM checklist.
• Statistical analyses, including correlation and logistic regression, were performed to identify factors associated with CLAIM adherence.

Main Results:

• A total of 501 articles were analyzed, with a median CLAIM score of 19 and an overall median compliance rate of 51.4%.
• Significant gaps in reporting were observed, particularly concerning annotation quality control, sample size determination, model robustness, failure analysis, and data/code availability.
• While overall adherence showed a weak positive trend with publication year, specific items like structured abstracts and data availability demonstrated significant improvements over time. Research objectives focused on image reconstruction, artifact reduction, multimodal imaging, and the use of public datasets were associated with lower reporting quality.

Conclusions:

• Current adherence to the CLAIM checklist in top medical imaging journals is insufficient, indicating a need for enhanced reporting standards.
• Collaborative efforts among researchers, editors, and reviewers are essential to strengthen the implementation and application of CLAIM.
• Improving adherence to CLAIM will enhance the quality, transparency, and reproducibility of AI research in medical imaging.