How measurement noise limits the accuracy of brain-behaviour predictions

Area of Science:

• Neuroimaging
• Individual Differences
• Biomarker Discovery

Background:

• Human neuroimaging research aims to predict behavioral phenotypes from brain data to understand individual differences and find clinical biomarkers.
• Generalizable and replicable brain-behavior prediction models require sufficient measurement reliability.
• Currently, prediction targets are often chosen based on scientific interest or data availability, not psychometric properties.

Purpose of the Study:

• To demonstrate the impact of low reliability in behavioral phenotypes on prediction performance.
• To investigate the relationship between measurement reliability, sample size, and the ability to link brain and behavior.

Main Methods:

• Analysis of simulated and empirical data from four large-scale datasets.
• Evaluation of prediction performance with varying levels of behavioral phenotype reliability.
• Utilizing data from 5000 participants in the UK Biobank to assess the effect of sample size.

Main Results:

• Low reliability in behavioral phenotypes can markedly limit the ability to establish brain-behavior associations.
• Increasing sample sizes from hundreds to thousands of participants only benefits prediction models when data reliability is high.
• Commonly observed reliability levels in behavioral phenotypes restrict the identification of meaningful brain-behavior links.

Conclusions:

• Measurement reliability is a critical factor for successful brain-behavior prediction in individual differences research.
• Psychometric considerations, particularly reliability, should be prioritized when selecting prediction targets in neuroimaging studies.
• Future research must emphasize psychometric rigor to advance the discovery of reliable neuroimaging biomarkers.