Clinical performance of automated machine learning: A systematic review

Arun James Thirunavukarasu ^1,2, Kabilan Elangovan ¹, Laura Gutierrez ¹

⁰Artificial Intelligence and Digital Innovation Research Group, Singapore Eye Research Institute, Singapore.

Annals of the Academy of Medicine, Singapore +

|

June 26, 2024

View abstract on PubMed

Summary

Automated machine learning (autoML) shows variable but competitive clinical performance across diverse applications. Further research is needed to enhance validation study quality for AI-driven healthcare advancements.

Area Of Science

Clinical applications of Artificial Intelligence (AI) and machine learning.
Medical informatics and data science.
Healthcare technology and innovation.

Background

Automated machine learning (autoML) aims to democratize AI model development by reducing technical barriers.
This review systematically evaluated autoML's clinical utility, platform capabilities, evidence quality, and performance benchmarks.
The study addresses the growing integration of AI in medical research and practice.

Approach

A comprehensive literature search was conducted across major databases (Cochrane Library, Embase, MEDLINE, Scopus).
Two researchers independently screened studies, extracted data, and assessed quality, with a third researcher for arbitration.
The review protocol was prospectively registered (PROSPERO CRD42022344427).

Key Points

Eighty-two studies featured 26 distinct autoML platforms, primarily in brain and lung disease research.
AutoML demonstrated variable performance (AUCROC: 0.35-1.00, F1: 0.16-0.99, AUPRC: 0.51-1.00), outperforming benchmarks in many trials.
AutoPrognosis and Amazon Rekognition showed strongest performance for unstructured and structured data, respectively, though reporting quality was poor.

Conclusions

AutoML platforms show promising clinical application and performance comparable to traditional methods.
Enhancing the quality of validation studies is crucial for reliable autoML implementation in healthcare.
Future directions include data-centric development and integration with large language models for self-improving AI systems.

Keywords:

AI artificial intelligence autoML automated machine learning deep learning machine learning