在大型视觉语言模型中评估和缓解关系幻觉
IEEE transactions on pattern analysis and machine intelligence
|February 3, 2026
概括
大视觉语言模型 (LVLMs) 与关系幻觉作斗争. 一个新的基准R-Bench和一种名为Region-Aware Alignment Mitigation (RA2M) 的方法通过专注于区域级调整来改善视觉关系的理解.
科学领域:
- 人工智能的人工智能
- 计算机视觉 计算机视觉
- 自然语言处理自然语言处理.
背景情况:
- 大视觉语言模型 (LVLMs) 显示出普遍存在的幻觉问题.
- 现有的研究主要针对对象幻觉,忽视了对视觉理解至关重要的对象间关系不准确性.
研究的目的:
- 引入R-Bench,这是一个用于评估 LVLMs中关系幻觉的新基准.
- 分析关系幻觉的原因,并提出缓解策略.
主要方法:
- 开发了R-Bench,使用图像级和实例级的问题来评估视觉关系.
- 分析了幻觉的原因,包括关系关系,主体关系和关系对象的共同发生,由长尾数据分布加剧.
- 拟议的区域意识调整缓解 (RA2M),以加强区域级图像-文本调整.
主要成果:
- 确定LVLM经常忽视视觉内容,过度依赖语言模型常识,特别是在空间推理中.
- 证明关系幻觉源于视觉数据集中的特定共发生模式.
- 展示了区域层面的调整可以显著缓解关系幻觉.
结论:
- 关系幻觉是LVLM的一个重大挑战,与对象幻觉不同.
- R-Bench为评估和推进 LVLM 在理解视觉关系方面的能力提供了一个关键工具.
- RA2M为改善LVLMs的事实准确性和视觉接地提供了一个有希望的方向.
相关概念视频
Language
917
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
917
Vision
60.1K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
60.1K
Relationship Formation
46.2K
What do you think is the single most influential factor in determining with whom you become friends and whom you form romantic relationships? You might be surprised to learn that the answer is simple: the people with whom you have the most contact. This most important factor is proximity. You are more likely to be friends with people you have regular contact with. For example, there are decades of research that shows that you are more likely to become friends with people who live in your dorm,...
46.2K
Positive Symptoms Schizophrenia: Hallucinations and Delusions
609
Schizophrenia is a complex psychiatric disorder characterized by a range of symptoms that significantly impact cognition, behavior, and emotional regulation. Among these, the positive symptoms stand out as they involve the addition or exaggeration of normal mental functions, deviating markedly from typical behavior and perception. Hallucinations and delusions are prominent positive symptoms, each profoundly affecting the individual's experience of reality.
Hallucinations
Hallucinations in...
Hallucinations
Hallucinations in...
609
Color Vision
1.5K
Color perception begins in the retina, the light-sensitive layer at the back of the eye. Two main theories explain how colors are seen: the trichromatic theory and the opponent-process theory. The trichromatic theory, proposed by Thomas Young in 1802 and extended by Hermann von Helmholtz in 1852, suggests that color vision is based on three types of cone receptors in the retina. These cones are sensitive to different but overlapping ranges of wavelengths corresponding to red, blue, and green.
1.5K
Ending Relationships
192
The dissolution of intimate relationships presents complex emotional and psychological challenges, particularly when emotional bonds are strong, the relationship is long-standing, and perceived alternatives are limited. This distress often intensifies in romantic breakups, where the initiator may experience greater turmoil than the rejected partner. Contributing factors include residual attachment, guilt over causing pain, and uncertainty about how to manage the situation. The stress is further...
192


