On the replicability of diffusion weighted MRI-based brain-behavior models

Replicability of anatomical and functional MRI-based inter-individual BWAS has been extensively discussed recently. This study reports a comprehensive evaluation of BWAS replicability based on structural connectomes (streamlines, FA, MD, RD, AD). Overall, 36%(21/58) of brain-phenotype associations were replicable in the HCP dataset with atleast one of the DWI metrics, and a discovery sample size n ≤ 425 (total sample size = discovery + replication samples). Temporally stable, trait-like phenotypes were found to be more replicable (50%, 16/32), than state-like measures (19%, 5/26). Streamline-based connectomes (SC) provided the highest replicability across all metrics (29% and 42% of phenotypes in the HCP and AOMIC datasets, respectively). In line with theoretical expectations, replicability was found to be directly related to effect size. Phenotypes that needed n > 400 discovery samples to replicate displayed very low effect sizes <2% variance). Effect size magnitudes replicated well with associations that explained more than ~5% variance, typically requiring a discovery n < 300. Our results suggest that trait-like phenotypes can show good replicability with moderate sample sizes and warrant that models that require n > 425 samples will necessarily display limited practical relevance due to their small predictive performance. Large sample sizes remain crucial for explainability and for assessing fairness and generalizability to new populations.