Permutations of functional magnetic resonance imaging classification may not be normally distributed

Area of Science:

• Neuroimaging
• Statistical analysis
• Machine learning

Background:

• Normality of distributions is crucial for statistical tests.
• Permutation distributions are common in neuroimaging analyses.
• The Central Limit Theorem (CLT) predicts normality under specific conditions, but this is under-explored in fMRI data.

Purpose of the Study:

• To assess the accordance of permutation distributions of classification accuracies with normality predicted by the CLT in fMRI data.
• To investigate the influence of different permutation schemes and data preprocessing steps on this accordance.
• To evaluate the reliability of cross-validation in fMRI analyses.

Main Methods:

• Used the Anderson-Darling test to evaluate normality of permutation distributions.
• Conducted a simulation study using fMRI data from visual stimulation tasks.
• Evaluated two scrambling schemes: permuting training/testing sets vs. permuting only testing set.
• Assessed the impact of motion correction and cross-validation folds.

Main Results:

• Permutation distributions often do not fit a normal distribution well.
• Normality improves when mean classification accuracies are averaged across multiple classifiers.
• Motion correction can negatively affect the approximation to normality.
• Strong dependencies exist across cross-validation folds and fMRI runs, potentially reducing reliability.
• Different permutation schemes yield different distributions and statistical significance assessments.

Conclusions:

• The assumption of normality for permutation distributions in fMRI is often violated.
• Averaging classifier accuracies can enhance normality, but careful consideration of permutation schemes and preprocessing is necessary.
• Cross-validation reliability in fMRI may be compromised by inherent data dependencies.
• Findings highlight the need for robust statistical methods in neuroimaging analyses.