Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Parallel Processing01:20

Parallel Processing

888
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
888
Cerebral Hemispheres01:05

Cerebral Hemispheres

3.5K
The human brain, a complex organ, is functionally divided into two cerebral hemispheres—left and right. These hemispheres are interconnected by a structure of paramount importance, the corpus callosum. This substantial bundle of neural fibers is not just a bridge between the hemispheres but a crucial element for the brain's comprehensive functioning. It enables efficient communication between the two hemispheres, allowing each side of the brain to control and receive sensory and motor...
3.5K
Multi-input and Multi-variable systems01:22

Multi-input and Multi-variable systems

502
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
502
Multiple Intelligences Theory01:20

Multiple Intelligences Theory

9.8K
Howard Gardner's theory of Multiple Intelligence proposes that there are nine distinct types of intelligence, each reflecting different ways of interacting with the world. Introduced in 1983 and expanded in subsequent years, Gardner's framework challenges the traditional notion of a single, generalized intelligence.
9.8K
Multicompartment Models: Overview01:14

Multicompartment Models: Overview

701
Multicompartment models are mathematical constructs that depict how drugs are distributed and eliminated within the body. They segment the body into several compartments, symbolizing various physiological or anatomical areas connected through drug transfer processes such as absorption, metabolism, distribution, and elimination.
These models offer a more comprehensive representation of drug behavior in the body than one-compartment models. They accommodate the complexity of drug distribution,...
701
Sensory Modalities01:15

Sensory Modalities

4.4K
Sensation typically is the process by which the sensory receptors and sense organs detect stimuli from the internal and external environment and transmit this information to the central nervous system for processing.
General senses refer to the broad category of sensory information detected by receptors in the body and can be further grouped into somatic and visceral senses. Somatic sensations include touch, pressure, temperature, and pain and are essential for navigating our environment and...
4.4K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

The properties of panels in global comics: frequency and size of 76 K panels in 1,030 comics from 144 countries.

Language resources and evaluation·2026
Same author

Neural dynamics of narrative structure-building in visual stories.

Cortex; a journal devoted to the study of the nervous system and behavior·2026
Same author

Three monkeys or one?: Continuity in the monkey emoji debate.

Acta psychologica·2026
Same author

Whoosh! visual depictions of direction, speed, and temporality: a corpus analysis of motion events in global comics.

Multimodal communication·2025
Same author

The Present and Future of Parallel Architectures of Language and Cognition.

Topics in cognitive science·2025
Same author

Understanding lightbulb moments: Meaning-making in visual morphology from comics and emoji.

Memory & cognition·2025
Same journal

Corrigendum to "Productivity matters for the neural processing of novel words, but not existing ones" Cognition Volume 274 (2026) 106593.

Cognition·2026
Same journal

Investigating the origins of partisanship: What motivates children to preferentially endorse their ingroups' claims?

Cognition·2026
Same journal

People make graded judgments about the inconceivable.

Cognition·2026
Same journal

The self as an image: Appearance and belief in visual representations of one's own face.

Cognition·2026
Same journal

Corrigendum to 'Consonant, vowel, and tone cues in early wordform recognition: Evidence from Cantonese-learning infants' [Cognition 275 (2026) 106624].

Cognition·2026
Same journal

Identifying distinct sources of whole number interference in children's decimal comparison: the role of numerical magnitude and inhibitory control.

Cognition·2026
See all related articles

Related Experiment Video

Updated: Mar 31, 2026

Cross-Modal Multivariate Pattern Analysis
13:51

Cross-Modal Multivariate Pattern Analysis

Published on: November 9, 2011

20.6K

A multimodal parallel architecture: A cognitive framework for multimodal interactions.

Neil Cohn1

  • 1Center for Research in Language, University of California, San Diego, United States.

Cognition
|October 23, 2015
PubMed
Summary
This summary is machine-generated.

This study proposes a new framework for understanding multimodal communication, integrating verbal, bodily, and visual elements. It expands theories of language to include complex interactions in visual narratives and graphic domains.

Keywords:
ComicsGestureLinguistic modelsMultimodalityNarrativeParallel architectureVisual language

More Related Videos

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties
12:55

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties

Published on: September 27, 2020

9.3K
Using the Race Model Inequality to Quantify Behavioral Multisensory Integration Effects
08:13

Using the Race Model Inequality to Quantify Behavioral Multisensory Integration Effects

Published on: May 10, 2019

6.9K

Related Experiment Videos

Last Updated: Mar 31, 2026

Cross-Modal Multivariate Pattern Analysis
13:51

Cross-Modal Multivariate Pattern Analysis

Published on: November 9, 2011

20.6K
Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties
12:55

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties

Published on: September 27, 2020

9.3K
Using the Race Model Inequality to Quantify Behavioral Multisensory Integration Effects
08:13

Using the Race Model Inequality to Quantify Behavioral Multisensory Integration Effects

Published on: May 10, 2019

6.9K

Area of Science:

  • Cognitive Science
  • Linguistics
  • Communication Studies

Background:

  • Human communication is inherently multimodal, with existing research focusing on speech-gesture and text-image semantics.
  • Visual narratives, such as comics, present unique challenges due to dual semantic guidance and complex sequential structures in both verbal and visual modalities.
  • Current models of multimodality and linguistic systems often struggle with the complexity arising from multiple, structured modalities.

Purpose of the Study:

  • To propose a theoretical framework for multimodal interactions by extending Jackendoff's parallel architecture for language.
  • To characterize multimodal interactions based on component cognitive structures: modality presence, grammatical structure (syntax, narrative), and semantic dominance.
  • To integrate multimodal interactions within existing frameworks of language and cognition, broadening the scope of the "linguistic" system.

Main Methods:

  • Expanding Jackendoff's (2002) parallel architecture for language.
  • Characterizing multimodal interactions by modality (verbal, bodily, visual), structure (grammatical, narrative), and semantic influence.
  • Integrating findings into a theoretical model for analyzing complex multimodal expressions.

Main Results:

  • A theoretical framework is presented for analyzing multimodal interactions across verbal, bodily, and graphic domains.
  • The framework characterizes interactions based on modality, grammatical structure, and semantic dominance.
  • The model expands the conceptual boundaries of the "linguistic" system to encompass multimodal phenomena.

Conclusions:

  • The proposed framework offers a comprehensive approach to understanding multimodal communication, particularly visual narratives.
  • This model integrates diverse communicative modalities into a unified cognitive and linguistic perspective.
  • The framework has potential applications in corpus analysis, experimental research, and educational strategies for multimodality.