Rethinking feature reproducibility in radiomics: the elephant in the dark

In radiomics, features are often linked to biomarkers and are generally expected to be reproducible, as reproducibility is considered a prerequisite for developing predictive models in clinical applications. However, this perspective overlooks feature interactions and may underestimate the potential value of nonreproducible features. Through experiments simulating a test-retest scenario, we demonstrate that even non-reproducible features can contribute significantly to predictive performance. Removing these features can lower model accuracy. These findings suggest that the emphasis on feature reproducibility should be reconsidered and that features should not be evaluated in isolation. Underlying information can be spread across multiple features. Focusing on individual features ignores feature interactions and may limit the model's predictive power. Ultimately, radiomics must prioritize prediction and clinical relevance. KEY POINTS: Feature reproducibility assessments often ignore feature interactions, overlooking predictive performance. Feature reproducibility depends on subjective thresholds, chosen metrics, and sample size. Nonreproducible features can be more predictive than reproducible ones. Predictive information may be distributed across multiple features rather than confined to individual ones.