Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results.

Area of Science:

• Neuroimaging
• Cognitive Neuroscience
• Machine Learning in Neuroscience

Background:

• Functional magnetic resonance imaging (fMRI) is crucial for in vivo cognitive studies.
• Functional connectivity (FC) and related metrics are increasingly used to predict phenotypes.
• Replicability issues in fMRI studies are a growing concern.

Approach:

• Demonstrate how treating longitudinal/contemporaneous scans of the same subject as independent inflates classification accuracy.
• Utilize the UK Biobank dataset to show identifiability can explain variance with fewer subjects.
• Replicate findings across four diverse datasets: UK Biobank, PNC, BSNIP, and a Fibromyalgia dataset.

Key Points:

• Unintentional accuracy inflation from 61% to 86% observed by not accounting for subject identifiability.
• Achieved similar predictive power with 50 subjects using identifiability as with 10,000 subjects without.
• Accuracy improvements ranged from 7% to 25% across datasets.
• Dynamic functional connectivity (dFC) allows this inflation even with single scans per subject.

Conclusions:

• Minor pipeline anomalies, like overlooking subject identifiability, can lead to inflated results.
• Features derived from inflated results may mislead future research.
• Awareness and correction of such biases are critical for robust neuroimaging research.