Clinical performance of automated machine learning: A systematic review

Area of Science:

• Clinical applications of Artificial Intelligence (AI) and machine learning.
• Medical informatics and data science.
• Healthcare technology and innovation.

Background:

• Automated machine learning (autoML) aims to democratize AI model development by reducing technical barriers.
• This review systematically evaluated autoML's clinical utility, platform capabilities, evidence quality, and performance benchmarks.
• The study addresses the growing integration of AI in medical research and practice.

Approach:

• A comprehensive literature search was conducted across major databases (Cochrane Library, Embase, MEDLINE, Scopus).
• Two researchers independently screened studies, extracted data, and assessed quality, with a third researcher for arbitration.
• The review protocol was prospectively registered (PROSPERO CRD42022344427).

Key Points:

• Eighty-two studies featured 26 distinct autoML platforms, primarily in brain and lung disease research.
• AutoML demonstrated variable performance (AUCROC: 0.35-1.00, F1: 0.16-0.99, AUPRC: 0.51-1.00), outperforming benchmarks in many trials.
• AutoPrognosis and Amazon Rekognition showed strongest performance for unstructured and structured data, respectively, though reporting quality was poor.

Conclusions:

• AutoML platforms show promising clinical application and performance comparable to traditional methods.
• Enhancing the quality of validation studies is crucial for reliable autoML implementation in healthcare.
• Future directions include data-centric development and integration with large language models for self-improving AI systems.