通过视觉和语言对人类行动的共享表示
Diana C Dima1, Sugitha Janarthanan2, Jody C Culham3
1Dept of Computer Science, Western University, London, Ontario, Canada; Vector Institute for Artificial Intelligence, Toronto, Ontario, Canada.
Neuropsychologia
|July 24, 2024
概括
人类行动的理解是通过视觉和语言在头脑中类似地组织的. 这种组织主要以行动目标为基础,反映了社会相关的目标并有助于沟通.
科学领域:
- 认知科学 认知科学
- 神经科学是一个神经科学.
- 计算语言学 计算语言学
背景情况:
- 人类擅长识别和沟通行为.
- 了解不同模式 (视觉,语言) 的行动是如何在精神上组织的,至关重要.
研究的目的:
- 为了研究人类行为的心理组织.
- 要确定这个组织是否在视觉和语言表示中共享.
- 探索行动目标和语义特征在这个组织中的作用.
主要方法:
- 从自然主义视频和句子中收集了人类行为的相似性判断.
- 在不同的抽象水平上测试了四种动作分类的计算模型.
- 利用语言模型嵌入来预测行为相似性.
主要成果:
- 在视频和句子模式中发现了行动表示的共享组织.
- 行动目标 (对象,人,自我) 是这种共享组织的主要决定因素.
- 语言模型嵌入成功预测了行为相似性,并捕获了行动目标信息.
结论:
- 行动概念在视觉和语言的头脑中组织得类似.
- 这种共享的组织主要是由社会相关的目标驱动的,特别是行动的目标.
- 语言模型可以有效地代表和预测跨模式的人类行为理解.
更多相关视频
09:27Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language
Published on: October 13, 2018
10.0K
07:43Author Spotlight: A Novel Setup to Conduct Naturalistic Laboratory Experiments with Real Human Actors in Scenarios
Published on: August 4, 2023
1.9K
相关概念视频
Language and Cognition
340
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
340
Vision
53.1K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
53.1K
Nonconscious Mimicry
4.5K
Nonconscious mimicry occurs when individuals alter their mannerisms to match the behaviors and expressions of those nearby, without intention.
4.5K
Parallel Processing
150
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
150
Visual System
564
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
564
Higher Mental Functions of the Brain: Language
779
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
779
