How ignoring physiological noise can bias the conclusions from fMRI simulation results

Neuroimaging researchers use simulation studies to validate their statistical methods because it is acknowledged that this is the most feasible way to know the ground truth of the data. The noise model used in these studies typically varies from a simple Gaussian distribution to an estimate of the noise distribution from real data. However, although several studies point out the presence of physiological noise in fMRI data, this noise source is currently lacking in simulation studies. Therefore, we explored the impact of adding physiological noise to the simulated data. For several experimental designs, fMRI data were generated under different noise models while the signal-to-noise ratio was kept constant. The sensitivity and specificity of a standard statistical parametric mapping (SPM) analysis were determined by comparing the known activation with the detected activation. We show that by including physiological noise in the data generation process, the simulation results in terms of sensitivity and specificity drop dramatically. Additionally, we used the new proposed simulation model to compare a standard SPM analysis against the method proposed by Cabella et al. (2009). The results indicate that the analysis of data containing no physiological noise yields a better performance of the SPM analysis. However, if physiological noise is included in the data, the sensitivity and specificity of the Cabella method are higher compared to the SPM analysis. Based on these results, we argue that the results of current simulation studies are likely to be biased, especially when analysis methods are compared using ROC curves.