Mind the gap: Performance metric evaluation in brain-age prediction.

Area of Science:

• Neuroscience
• Artificial Intelligence
• Biostatistics

Background:

• Neuroimaging-derived data is increasingly used for estimating brain age and assessing brain health.
• Machine learning models can predict age from brain characteristics, but reported accuracy varies widely across studies.
• Standard performance metrics for these models include Pearson's correlation coefficient (r), R-squared, Root Mean Squared Error (RMSE), and Mean Absolute Error (MAE).

Purpose of the Study:

• To investigate how factors like age range, sample size, and age-bias correction affect the performance metrics of age prediction models.
• To assess the reliability and comparability of different performance metrics across various study conditions.
• To provide guidance on interpreting model performance for accurate brain age estimation.

Main Methods:

• Age prediction was performed on two population-based neuroimaging datasets.
• The impact of varying age ranges and sample sizes on model performance was evaluated.
• The influence of age-bias correction on standard performance metrics (r, R², RMSE, MAE) was assessed.

Main Results:

• Performance metrics (r, R²) were lower in datasets with narrower age ranges.
• RMSE and MAE were also lower in narrower age ranges due to predictions clustering around the mean age.
• Model performance metrics improved with larger sample sizes across different age ranges.
• Age-bias corrected metrics indicated high accuracy even for models with initially poor performance, highlighting the importance of prediction variance.

Conclusions:

• Performance metrics for neuroimaging-based age prediction models are highly dependent on cohort and study-specific data characteristics.
• Direct comparison of performance metrics across different studies is not reliable due to variations in methodology and data.
• While age-bias corrected metrics suggest high accuracy, examining uncorrected metrics is crucial for understanding model attributes like prediction variance.