Availability of Evidence for Predictive Machine Learning Algorithms in Primary Care: A Systematic Review

Area of Science:

• Health Informatics
• Artificial Intelligence in Healthcare
• Primary Care Research

Background:

• Primary healthcare faces challenges from aging populations and staff shortages.
• Predictive machine learning (ML) offers potential solutions but faces concerns regarding transparency and reporting of validation and implementation effectiveness.
• There is a need to systematically assess the evidence base for ML algorithms in primary care.

Purpose of the Study:

• To systematically identify predictive ML algorithms used in primary care.
• To evaluate the public availability of evidence across the AI life cycle for these algorithms.
• To assess adherence to reporting guidelines for predictive ML algorithms in primary care.

Main Methods:

• A comprehensive literature search was conducted across multiple databases (PubMed, Embase, Web of Science, etc.) from January 2000 to July 2023.
• Searches included peer-reviewed literature and registration databases for FDA- and CE-marked predictive ML algorithms.
• Evidence availability was assessed against the Dutch AI predictive algorithm (AIPA) guideline requirements across all AI life cycle phases.

Main Results:

• 43 predictive ML algorithms were identified; 25 were commercially available (FDA/CE-marked).
• Most algorithms focused on cardiovascular diseases and diabetes, with 81% published in the last 5 years.
• Evidence availability was lowest for algorithm preparation (19%) and impact assessment (30%), with peer-reviewed literature showing higher availability (45%) than regulatory databases (29%).

Conclusions:

• There is a critical need to enhance the availability of evidence concerning the quality criteria of predictive ML algorithms in primary care.
• Implementing the Dutch AIPA guideline can promote transparent and standardized reporting.
• Improved reporting can foster end-user trust and facilitate the large-scale adoption of these technologies.