The pitfalls of fixed-ratio data splitting in radiomics model performance evaluation

Area of Science:

• Medical Imaging
• Radiology
• Data Science

Background:

• Radiomics has grown significantly over the last decade, with numerous publications.
• Radiomics offers great potential in medical diagnostics and prognostics.
• Current radiomics research faces challenges in model validation.

Purpose of the Study:

• To highlight the limitations of current radiomics model validation methods.
• To advocate for the adoption of more robust validation techniques.
• To ensure reliable and reproducible radiomics model performance.

Main Methods:

• Analysis of common practices in radiomics model validation.
• Identification of issues associated with single-center datasets and fixed-ratio splits.
• Review of alternative cross-validation strategies.

Main Results:

• Most radiomics studies utilize single-center datasets.
• Fixed-ratio data splitting introduces variability and reduces model reliability.
• The randomness in data splitting impacts performance evaluation.

Conclusions:

• The fixed-ratio holdout method is insufficient for robust radiomics model validation.
• Implementing rigorous cross-validation techniques is crucial.
• Adopting advanced validation methods will enhance the clinical utility of radiomics.