Robust is not necessarily reliable: From within-subjects fMRI contrasts to between-subjects comparisons

Advances in cognitive and affective neuroscience come largely from within-subjects comparisons, in which the functional significance of neural activity is determined by contrasting two or more experimental conditions. Clinical and social neuroscience studies have attempted to leverage between-subject variability in such condition differences to better understand psychopathology and other individual differences. Shifting from within-to between-subjects comparisons requires that measures have adequate internal consistency to function as individual difference variables. This is particularly relevant for difference scores-which have lower reliability. The field has assumed reasonable internal consistency of neural measures based on consistent findings across studies (i.e., if a within-subject difference in neural activity is robust, then it must be reliable). Using one of the most common fMRI paradigms in the clinical neuroscience literature (i.e., a face- and shape-matching task), in a large sample of adolescents (N = 139) we replicate a robust finding: amygdala activation is greater for faces than shapes. Moreover, we demonstrate that the internal consistency of the amygdala in face and shape blocks was excellent (Spearman-Brown corrected reliability [SB] > .94). However, the internal consistency of the activation difference between faces and shapes was nearly zero (SB = -.06). This reflected the fact that the amygdala response to faces and shapes was highly correlated (r = .97) across individuals. Increased neural activation to faces versus shapes could not possibly function as an individual difference measure in these data-illustrating how neural activation can be robust within subjects, but unreliable as an individual difference measure. Strong and reproducible condition differences in neural activity are not necessarily well-suited for individual differences research-and neuroimaging studies should always report the internal consistency of, and correlations between, activations used in individual differences research.