Words and Worlds Both: Dynamic Effects of Distributional and Sensorimotor Information in Semantic Processing

Area of Science:

• Cognitive Neuroscience
• Computational Linguistics
• Psychology

Background:

• Semantic memory research debates the roles of sensorimotor experience vs. language data in concept representation.
• Understanding how these different information sources interact is crucial for explaining conceptual processing.

Purpose of the Study:

• To investigate the relative contributions of sensorimotor and distributional information to behavioral and neural responses in a conceptual task.
• To determine if both experience-based and language-based semantic information are necessary for accurate conceptual representation.

Main Methods:

• Recorded electroencephalography (EEG) and reaction times (RTs) from healthy adults performing a concept-property verification task.
• Developed computational models using semantic distance measures from language corpora (distributional) and crowd-sourced norms (sensorimotor).
• Employed statistical model comparison to assess the predictive power of distributional, sensorimotor, and hybrid models on behavioral and neural data.

Main Results:

• Both distributional and sensorimotor semantic distance measures uniquely explained variance in behavioral (RTs) and neural (EEG) data.
• EEG analysis across 100-ms intervals showed distinct temporal dynamics for true vs. false trials.
• Early sensorimotor activation was observed for both true and false trials, suggesting its rapid involvement.

Conclusions:

• Findings support dynamic hybrid accounts of semantic memory, indicating that both sensorimotor and distributional information are actively used.
• Linguistic context and task demands dynamically modulate the recruitment of different information sources in semantic processing.
• This research clarifies the interplay between embodied experience and abstract linguistic knowledge in forming conceptual representations.