语言与视觉:一项关于接地词和句子嵌入的研究
Hassan Shahmohammadi1, Maria Heitmeier2, Elnaz Shafaei-Bajestan2
1University of Tübingen, Tübingen, Germany. hassan.shahmohammadi@uni-tuebingen.de.
Behavior research methods
|December 20, 2023
概括
这项研究引入了一个基于视觉语言的计算模型,对具体和抽象的单词都有好处. 该模型通过平衡文本数据与视觉信息来增强词语嵌入,以更好地表示.
科学领域:
- 认知科学 认知科学
- 自然语言处理自然语言处理.
- 计算机视觉 计算机视觉
背景情况:
- 视觉中的接地语言旨在通过将视觉感知与文本集成来创建认知上可信的表示.
- 当前的挑战包括优化文本和体现表达之间的平衡,并确定视觉接地对抽象与具体词语的有效性范围.
- 视觉知识在多大程度上提高了嵌入质量仍然是一个活跃的研究问题.
研究的目的:
- 调查视觉接地对抽象和具体单词的有效性.
- 提出一种最优的方法来弥合语言表现中文本和视觉之间的差距.
- 确定感知图像知识的优势,以获得高质量的文字嵌入.
主要方法:
- 开发了一个计算接地模型,以使预先训练的词嵌入与视觉信息保持一致.
- 该模型保留了文字分布统计数据,同时结合了视觉接地.
- 采用学习的对齐策略,间接地接地看不见的和抽象的单词.
主要成果:
- 视觉接地证明了对具体和抽象单词的好处,支持抽象概念的间接理论.
- 拟议的模型有效地平衡了语言和视觉,改善了词语嵌入.
- 像BERT这样的上下文嵌入式在适度的语料库大小上训练时显示出优势.
结论:
- 视觉接地是一种有价值的技术,适用于广泛的词汇,包括抽象概念.
- 开发的模型提供了一种有效的方法,将视觉信息集成到语言表示中.
- 这些发现对改善语言模型和理解语言学习中的认知过程有影响.
相关概念视频
Vision
53.5K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
53.5K
Language and Cognition
349
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
349
Language
231
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
231
Higher Mental Functions of the Brain: Language
876
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
876
Language Development
368
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
368
Components of Language
285
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
285


