One Algorithm May Not Fit All: How Selection Bias Affects Machine Learning Performance.

Area of Science:

• Radiology
• Medical Imaging
• Artificial Intelligence

Background:

• Machine learning (ML) algorithms demonstrate high diagnostic accuracy on radiologic images in initial studies.
• However, performance often declines when ML algorithms are deployed in new clinical environments with different image datasets.

Purpose of the Study:

• To highlight the critical issue of selection bias in the development of ML algorithms for medical imaging.
• To equip radiologists with the knowledge to critically assess ML literature for potential biases and understand real-world applicability.

Main Methods:

• Review of existing literature on ML in medical imaging and clinical epidemiology.
• Discussion of selection bias as a factor influencing ML algorithm performance and generalizability.

Main Results:

• Selection bias in training data is a primary reason for the performance gap between ML algorithms in research and clinical practice.
• This bias, well-understood in clinical epidemiology, is under-addressed in current ML medical imaging research.

Conclusions:

• Selection bias significantly impacts the reliability and generalizability of ML algorithms in radiology.
• Radiologists must critically evaluate ML studies for selection bias to ensure safe and effective clinical implementation.