Do We Have Enough Information? Assessing AI Clinical Decision Support Systems for Implementation in Primary Care

Area of Science:

• Health Informatics
• Artificial Intelligence in Healthcare
• Clinical Decision Support Systems

Background:

• AI clinical decision support solutions (AI-CDSS) are increasingly prevalent in healthcare.
• Evaluating the transparency and rigor of these AI-CDSS is crucial for safe and effective implementation.
• Existing AI-CDSS lack standardized reporting on key aspects like knowledge base, AI methodology, and data privacy.

Purpose of the Study:

• To conduct an environmental scan of commercially available AI-CDSS.
• To assess the transparency and quality of information provided regarding knowledge base, AI methodology, and privacy.
• To identify gaps hindering reliable evaluation and informed implementation of AI-CDSS.

Main Methods:

• Environmental scan of commercial AI-CDSS.
• Analysis of vendor-disclosed information across three domains: knowledge base, AI methodology, and privacy.
• Assessment of information rigor, alignment with quality standards, and data privacy details.

Main Results:

• Over 50% of vendors disclosed some knowledge base information, but few showed rigorous appraisal or adherence to evidence-based guidelines.
• Transparency regarding AI methodology was limited, with most citing proprietary algorithms and rarely detailing training data.
• Privacy information was more frequently reported but generally high-level, lacking specifics on compliance, data storage, or secondary use.

Conclusions:

• Significant gaps exist in the transparency of AI-CDSS regarding their knowledge base, AI methodology, and data privacy.
• These deficiencies create obstacles for decision-makers in reliably evaluating AI-CDSS.
• Standardized, transparent reporting is needed to support clinicians in the responsible and informed implementation of AI-CDSS.