Event-related potentials during discourse-level semantic integration of complex pictures

This study examined event-related potentials (ERPs) elicited in response to semantic processing of non-verbal stories. ERPs were recorded from 29 scalp electrodes on 16 participants while they viewed series of complex gray-scale pictures, each of which relayed a simple story. The final picture of each story was either congruous or incongruous with the preceding context. Participants made delayed meaningfulness judgments for each story. Averaged ERPs time-locked to the onset of the final picture were more negative for incongruous than congruous pictures. Two distinct components were sensitive to congruency. The first component peaked at approximately 325 ms (N300) and was distributed over central and frontal sites. The second component peaked at approximately 500 ms and also had a centro-frontal maximum but was more widespread than the earlier component (anterior N400). The distinct scalp topographies of these two negativities provide strong evidence that the N300 and N400 are separate and distinguishable components. Furthermore, the presence of the N300 in this exclusively pictorial task suggests that the N300 is specific to the semantic processing of non-verbal stimuli and is not due to linguistic mediation. This study also revealed that the N400 can be modulated by discourse-level coherence manipulations with pictures. Finally, the different patterns of ERP effects observed during the semantic processing of verbal and non-verbal information suggest that non-identical neuronal sources, and thus non-identical representational systems, are accessed by these different types of materials. These findings strongly support at least partial modularity of semantic representations and processing mechanisms in the human brain.