An electrophysiological investigation of co-referential processes in visual narrative comprehension

Visual narratives make use of various means to convey referential and co-referential meaning, so comprehenders must recognize that different depictions across sequential images represent the same character(s). In this study, we investigated how the order in which different types of panels in visual sequences are presented affects how the unfolding narrative is comprehended. Participants viewed short comic strips while their electroencephalogram (EEG) was recorded. We analyzed evoked and induced EEG activity elicited by both full panels (showing a full character) and refiner panels (showing only a zoom of that full panel), and took into account whether they preceded or followed the panel to which they were co-referentially related (i.e., were cataphoric or anaphoric). We found that full panels elicited both larger N300 amplitude and increased gamma-band power compared to refiner panels. Anaphoric panels elicited a sustained negativity compared to cataphoric panels, which appeared to be sensitive to the referential status of the anaphoric panel. In the time-frequency domain, anaphoric panels elicited reduced 8-12 Hz alpha power and increased 45-65 Hz gamma-band power compared to cataphoric panels. These findings are consistent with models in which the processes involved in visual narrative comprehension partially overlap with those in language comprehension.