远远看清:缓解MLLM的幻觉,注意因果解码.
Feilong Tang1,2, Chengzhi Liu3, Zhongxing Xu1
1Monash University.
Proceedings. IEEE Computer Society Conference on Computer Vision and Pattern Recognition
|September 15, 2025
概括
本研究介绍了FarSight,这是一个新的解码策略,用于减少多式联络大型语言模型 (MLLMs) 中的幻觉. 远视优化因果面具,以改善令牌交互和增强上下文推断,以更好地视觉回答问题.
科学领域:
- 人工智能的人工智能
- 计算机视觉 计算机视觉
- 自然语言处理自然语言处理.
背景情况:
- 多模式大语言模型 (MLLMs) 在视觉问题回答方面表现出色,但在幻觉方面扎.
- 在MLLM中,幻觉被分为初始类型和雪球类型,这些类型是由于代币交互不足而产生的.
- 语境信息对于准确的MLLM推断至关重要,但由于异常符号影响,经常被忽视.
研究的目的:
- 提出一个新的解码策略,FarSight,以减轻MLLM的幻觉.
- 通过改善多式联络令牌之间的信息传播来增强上下文推断.
- 通过优化因果面具来减少异常令牌的注意干扰.
主要方法:
- 在因果推理的启发下,FarSight利用因果面具来建立多式联络令牌之间的信息传播.
- 在因果面具中的注意力注册结构动态地分配注意力,以防止转移到异常代币.
- 一种位置感知编码方法,具有减小的掩盖率,可以提高对先前令牌的注意力,特别是在视频任务中.
主要成果:
- 在各种基准测试中,FarSight 显著减少了多模式大语言模型中的幻觉.
- 插即用策略在基于图像和视频的视觉问答任务中表现出有效性.
- 实验证实了FarSight通过解决异常符号干扰来增强上下文推断的能力.
结论:
- 远视提供了一种多功能和有效的解决方案,以减轻MLLM的幻觉.
- 优化因果面具和令牌传播是提高MLLM可靠性的关键.
- 拟议的方法提高了多式联运人工智能系统的准确性和可靠性.
更多相关视频
07:30HSV-Mediated Transgene Expression of Chimeric Constructs to Study Behavioral Function of GPCR Heteromers in Mice
Published on: July 9, 2016
7.6K
08:45Mapping Cortical Dynamics Using Simultaneous MEG/EEG and Anatomically-constrained Minimum-norm Estimates: an Auditory Attention Example
Published on: October 24, 2012
15.2K
相关概念视频
01:06Altered States of Awareness
1.0K
Altered states of consciousness represent significant deviations from one's normal mental state. These deviations can range from subtle changes in awareness to profound transformations in perception, thought processes, and sensory experiences. Altered states of consciousness can be triggered by various factors, including drug use, meditation, hypnosis, illness, or even intense fatigue.
The ingestion of substances like stimulants or hallucinogens leads to chemical alterations in the brain...
The ingestion of substances like stimulants or hallucinogens leads to chemical alterations in the brain...
1.0K
01:27Hallucinogens and Psychedelics
616
Hallucinogens are psychoactive substances that profoundly alter perceptual experiences, generating unreal visual and sensory images. Often referred to as psychedelic drugs — a term derived from the Greek words "psyche" (mind) and "delos" (revealing) — these substances include marijuana and lysergic acid diethylamide (LSD), among others. These drugs vary in intensity and effects.
Marijuana, derived from the dried leaves and flowers of the hemp plant, contains...
Marijuana, derived from the dried leaves and flowers of the hemp plant, contains...
616