False-positive neuroimaging: Undisclosed flexibility in testing spatial hypotheses allows presenting anything as a...

Area of Science:

• Neuroimaging
• Cognitive Neuroscience
• Brain Imaging Analysis

Background:

• Neuroimaging studies frequently treat anatomical regions as single units for hypothesis testing.
• This approach creates ambiguity between voxel-level data and region-level inferences, leading to a "model degrees of freedom" that hinders reproducibility.
• Claims of replication in functional Magnetic Resonance Imaging (fMRI) studies often lack quantitative spatial evidence.

Purpose of the Study:

• To highlight the discrepancy between analytical units and inferential units in neuroimaging hypothesis testing.
• To identify the extent of analytical flexibility and its impact on the reproducibility of findings in published fMRI studies.
• To advocate for the adoption of quantitative spatial models for more rigorous hypothesis testing in neuroimaging.

Main Methods:

• Surveyed 135 functional Magnetic Resonance Imaging studies claiming replication.
• Analyzed the reporting of quantitative evidence for replication, such as activation peaks and coordinate-based models.
• Assessed the spatial proximity of reported activation peaks in purported replication studies.

Main Results:

• 42.2% of studies lacked quantitative replication evidence (e.g., activation peaks).
• Only 14.1% used precise coordinate-based or a priori pattern-based models.
• 42.9% of replicated findings had peak coordinates >15mm from original findings, indicating different activated brain locations.

Conclusions:

• Region-level hypothesis testing in neuroimaging is often too flexible and qualitative, undermining reproducibility.
• Quantitative spatial models and tests (e.g., permutation tests, Bayesian MANOVA, pattern-based models) are recommended.
• Adopting these methods will enable precise, falsifiable spatial hypotheses, fostering a more cumulative neuroimaging science.