AI Quality Standards in Health Care: Rapid Umbrella Review

Area of Science:

• Healthcare Informatics
• Artificial Intelligence in Medicine
• Systematic Review Methodology

Background:

• The proliferation of artificial intelligence (AI) in healthcare research necessitates robust evaluation standards.
• Existing AI quality standards aim to assess the rigor of healthcare AI studies.

Purpose of the Study:

• To conduct a rapid umbrella review of AI quality standards in healthcare systematic reviews.
• To examine the application and effectiveness of AI quality standards in published literature.

Main Methods:

• A modified Joanna Briggs Institute umbrella review methodology was employed.
• A focused search on MEDLINE and Google Scholar identified English-language systematic reviews mentioning AI and health.
• Included reviews were analyzed for the use of AI quality standards and identified issues.

Main Results:

• 33 systematic reviews (2020-2022) were synthesized, covering diverse AI applications in healthcare.
• AI quality standards were applied in only 39% of reviews and 14% of primary studies.
• Key quality aspects like transparency, ethics, and privacy were infrequently addressed, highlighting a need for standardization.

Conclusions:

• Healthcare AI quality standards are underutilized despite their increasing availability.
• Adoption of evolving AI quality standards is crucial for improving the reliability and trustworthiness of healthcare AI research.
• Addressing ethical, societal, and regulatory implications is paramount for future AI studies.