Improved Estimation of Correlation Accuracy for Machine Learning Brain-Phenotype Associations.

Area of Science:

• Neuroscience
• Machine Learning
• Biostatistics

Background:

• Machine learning (ML) is crucial for analyzing brain-phenotype associations and individual prediction from neuroimaging data.
• Current methods using Pearson's correlation to estimate model accuracy, specifically maximum achievable predictive accuracy (MAPA), are unreliable, potentially requiring millions of samples.

Purpose of the Study:

• To formally define MAPA and demonstrate the bias of Pearson's estimator for this quantity.
• To develop and validate a novel semiparametric (double machine learning) estimator for MAPA that provides accurate estimates and valid confidence intervals.

Main Methods:

• Formal definition of Maximum Achievable Predictive Accuracy (MAPA).
• Development of a semiparametric (double machine learning) one-step estimator.
• Validation using the Reproducible Brain Charts dataset, analyzing neuroimaging data with age and psychopathology phenotypes.

Main Results:

• Pearson's correlation estimator is biased for MAPA, and its confidence intervals are inadequate.
• The proposed double machine learning estimator shows reduced bias in estimating brain-phenotype associations from neuroimaging data.
• MAPA for psychopathology factor scores using neuroimaging was not superior to using demographic and nuisance covariates alone.

Conclusions:

• The developed double machine learning estimator offers a more reliable method for assessing MAPA in neuroimaging studies.
• Neuroimaging data alone may have limited predictive power for psychopathology factor scores compared to basic covariates.
• Accurate estimation of MAPA is critical for advancing ML applications in neuroscience research.