Modality Effects Examined by Means of an Online Sentence-Picture Comparison Task

Are oral sentences accompanied by pictures easier to understand than written sentences accompanied by the same pictures? This question-intensely discussed for more than two decades in educational, psychological, and psycholinguistic research in terms of modality effect in multimedia learning, split-attention effect, or visuospatial load effect-was examined by means of an online sentence-picture comparison task, with participants reading or listening to short sentences accompanied by pictures. Sentences and pictures referred to a conceptually and linguistically basic universe that was constructed from a limited number of familiar objects and characteristics. Mixed repeated-measures ANOVAs were calculated separately for simultaneously and sequentially presented sentence-picture versions, with modality (oral, written), picture complexity (complex, simple), and sentence-picture compatibility (compatible, incompatible) as within-subject factors, and age (6th-graders, adults) as between-subjects factor. The experiment was based on reaction time and acuity measures. The online sentence-picture comparison task requires participants to take note of both components (verbal and pictorial information). The presence of two age groups made it possible to examine modality effects from a developmental point of view by suggesting that learners' written comprehension and monitoring skills at some point equal and then surpass their oral comprehension and monitoring skills. The experiment showed the necessity to interpret modality main effects in the context of their two-way and three-way interactions with other variables since modality effects taken alone do not tell the whole story. The concepts of split attention, temporal contiguity, and element interactivity were discussed in this context.