Inconsistent Partitioning and Unproductive Feature Associations Yield Idealized Radiomic Models

Area of Science:

• Medical imaging analysis
• Machine learning in healthcare
• Quantitative imaging biomarkers

Background:

• Radiomics extracts quantitative features from medical images for clinical prediction.
• Existing radiomic machine learning (ML) studies frequently lack complete methodology and external validation.
• Performance-inflating methodologic flaws are prevalent in published radiomic ML research.

Purpose of the Study:

• To identify and characterize common methodologic errors in radiomic ML studies.
• To assess the prevalence and statistical impact of these flaws.
• To demonstrate how these errors can inflate reported accuracy and lead to false conclusions.

Main Methods:

• Systematic review of radiomic ML publications for methodologic flaws.
• Simulation studies reproducing common flaws using random data.
• Analysis of the impact of flawed partitioning and feature association on model performance.

Main Results:

• Two primary error categories identified: inconsistent partitioning and unproductive feature associations.
• Inconsistent partitioning inflated ML accuracy by 1.4 times in simulations.
• Correcting flawed methods resulted in performance near random chance (AUC 0.5).
• Simulations showed spurious associations between radiomic features and gene sets, implying false causality.

Conclusions:

• Methodologic flaws significantly undermine the validity of radiomic ML studies.
• Inconsistent partitioning and unproductive feature associations are critical issues.
• External validation and rigorous methodology are essential for reliable radiomic ML.
• A review template is provided to help avoid these common pitfalls.