Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Components of Language01:24

Components of Language

496
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
496
Parallel Processing01:20

Parallel Processing

331
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
331
Higher Mental Functions of the Brain: Language01:10

Higher Mental Functions of the Brain: Language

1.7K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
1.7K
Language and Cognition01:27

Language and Cognition

498
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
498
Parallel Resonance01:23

Parallel Resonance

291
The parallel RLC circuit is an arrangement where the resistor (R), inductor (L), and capacitor (C) are all connected to the same nodes and, as a result, share the same voltage across them. The parallel RLC circuit is analyzed in terms of admittance (Y), which reflects the ease with which current can flow. The admittance is given by:
291
Multi-input and Multi-variable systems01:22

Multi-input and Multi-variable systems

192
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
192

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

The properties of panels in global comics: frequency and size of 76 K panels in 1,030 comics from 144 countries.

Language resources and evaluation·2026
Same author

Neural dynamics of narrative structure-building in visual stories.

Cortex; a journal devoted to the study of the nervous system and behavior·2026
Same author

Three monkeys or one?: Continuity in the monkey emoji debate.

Acta psychologica·2026
Same author

Whoosh! visual depictions of direction, speed, and temporality: a corpus analysis of motion events in global comics.

Multimodal communication·2025
Same author

The Present and Future of Parallel Architectures of Language and Cognition.

Topics in cognitive science·2025
Same author

Understanding lightbulb moments: Meaning-making in visual morphology from comics and emoji.

Memory & cognition·2025
Same journal

Scale, trust, and the digital divide: a systematic review of AI and ML for agricultural applications.

Frontiers in artificial intelligence·2026
Same journal

Beyond uncertainty in modern active learning for trustworthy AI.

Frontiers in artificial intelligence·2026
Same journal

Eco-FinOps: a causal-agentic framework for energy-efficient and explainable cloud cost optimization.

Frontiers in artificial intelligence·2026
Same journal

Multimodal graph neural network with large language models for node and link prediction.

Frontiers in artificial intelligence·2026
Same journal

Efficient representation of boolean decision structures through Boolean function optimization.

Frontiers in artificial intelligence·2026
Same journal

Structural impact of non-IID heterogeneity on federated behavioral anomaly detection in IoT and IoMT systems.

Frontiers in artificial intelligence·2026
See all related articles

Related Experiment Video

Updated: Oct 6, 2025

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks
08:32

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks

Published on: September 5, 2019

5.7K

Remarks on Multimodality: Grammatical Interactions in the Parallel Architecture.

Neil Cohn1, Joost Schilperoord1

  • 1Department of Communication and Cognition, Tilburg School of Humanities and Digital Sciences, Tilburg University, Tilburg, Netherlands.

Frontiers in Artificial Intelligence
|January 21, 2022
PubMed
Summary
This summary is machine-generated.

Multimodal communication integrates language with other behaviors like gesture and visuals. This study reveals that these interactions stem from unified substructures within a cognitive architecture, not just whole behaviors.

Keywords:
codeswitchinggrammarlinguistic theorymultimodalityparallel architecture

More Related Videos

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language
09:27

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language

Published on: October 13, 2018

10.2K
Real-Time Proxy-Control of Re-Parameterized Peripheral Signals using a Close-Loop Interface
11:54

Real-Time Proxy-Control of Re-Parameterized Peripheral Signals using a Close-Loop Interface

Published on: May 8, 2021

4.7K

Related Experiment Videos

Last Updated: Oct 6, 2025

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks
08:32

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks

Published on: September 5, 2019

5.7K
Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language
09:27

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language

Published on: October 13, 2018

10.2K
Real-Time Proxy-Control of Re-Parameterized Peripheral Signals using a Close-Loop Interface
11:54

Real-Time Proxy-Control of Re-Parameterized Peripheral Signals using a Close-Loop Interface

Published on: May 8, 2021

4.7K

Area of Science:

  • Cognitive Science
  • Linguistics
  • Communication Studies

Background:

  • Traditional models of linguistic competence often overlook the complexity of multimodal communication.
  • Speech, gesture, and visuals are frequently treated as indivisible units in multimodal messages.

Purpose of the Study:

  • To propose that multimodality is defined by interactions of similar substructures across expressive behaviors, not whole behaviors.
  • To introduce a unified cognitive architecture, Jackendoff's Parallel Architecture, comprising modality, meaning, and grammar, that underlies multimodal interactions.
  • To analyze interactions between grammars based on symmetry and allocation, including substitutive allocation.

Main Methods:

  • Analysis of interactions within and across vocal, bodily, and graphic modalities.
  • Examination of how substructures within the tripartite architecture (modality, meaning, grammar) interact.
  • Investigation of grammar interactions, focusing on symmetry and allocation, particularly substitutive allocation.

Main Results:

  • Multimodal interactions are characterized by correspondences in time (e.g., speech-gesture) and space (e.g., writing-pictures).
  • Multimodal meaning-making involves balancing semantic weight across different modalities.
  • Substitutive allocation, where expressions from one grammar are inserted into another, is observed across natural modalities and in unimodal contexts like codeswitching.

Conclusions:

  • Unimodal and multimodal expressions emerge from interactive states within a unified cognitive architecture.
  • The findings suggest a reconsideration of the concept of the 'language faculty' itself.
  • This framework provides a new perspective on how language integrates with other communicative behaviors.