How Robust Are fMRI- and EEG-Based Representational Similarity Analysis?

Area of Science:

• Neuroimaging analysis
• Cognitive neuroscience
• Data science

Background:

• Neuroimaging researchers face numerous pipeline choices.
• Pipeline reliability is often not apparent during analysis.
• Reanalysis with alternative pipelines can reveal result robustness.

Purpose of the Study:

• To adapt Specification Curve Analysis (SCA) for neuroimaging.
• To quantitatively assess the robustness of fMRI and EEG results.
• To evaluate the reliability of conclusions drawn from neuroimaging data.

Main Methods:

• Adapted Specification Curve Analysis (SCA) from psychology.
• Reanalyzed the THINGS dataset using various pipeline configurations.
• Developed a decision tree for identifying robust specifications.

Main Results:

• EEG-based representational similarity analysis (RSA) conclusions were robust.
• fMRI-based RSA conclusions were not robust to alternative specifications.
• Even the most robust fMRI specifications yielded fragile conclusions.

Conclusions:

• SCA is a valuable tool for assessing neuroimaging result reliability.
• SCA should be widely applied to event-related fMRI analysis.
• Modifications to SCA specifications may be needed for diverse datasets.