Feature selection leads to divergent neurobiological interpretations of brain-based machine learning biomarkers

A central objective in human neuroimaging is to understand the neurobiology underlying cognition and mental health. Machine learning models trained on neuroimaging data are increasingly used as tools for predicting behavioural phenotypes, enhancing precision medicine and improving generalizability compared with traditional MRI studies. However, the high dimensionality of brain connectivity data makes model interpretation challenging. Prevailing practices rely on selecting features and, implicitly, interpreting identified feature networks as uniquely representative of a given phenotype while overlooking others. Despite its widespread use, how univariate feature selection balances the trade-off between simplification for optimizing modelling and oversimplification that misrepresents true neurobiology remains understudied. Here, using four large-scale neuroimaging datasets spanning over 12,000 participants and 13 outcomes, we demonstrate that edges discarded by feature selection can achieve significant prediction accuracies while yielding different neurobiological interpretations. These results are observed across cognitive, developmental and psychiatric phenotypes, extend to both functional connectivity (functional MRI) and structural (diffusion tensor imaging) connectomes, and remain evident in external validation. They suggest that focusing on only the top features may simplify the neurobiological bases of brain-behaviour associations. Such interpretations present only the tip of the iceberg when certain disregarded features may be just as meaningful, potentially contributing to ongoing issues surrounding reproducibility within the field. More broadly, our results reinforce that subtle brain-wide signals should not be ignored.