迷你双子:挖掘多模式视觉语言模型的潜力
IEEE transactions on pattern analysis and machine intelligence
|November 26, 2025
概括
迷你双子通过改进高分辨率的视觉令牌和使用高质量的数据来增强多模式视觉语言模型 (VLM). 这种框架提高了图像理解,推理和生成,超过了现有的模型.
科学领域:
- 人工智能的人工智能
- 计算机视觉 计算机视觉
- 自然语言处理自然语言处理.
背景情况:
- 视觉语言模型 (VLM) 是有前途的,但在复杂的任务中落后于先进的专有模型.
- 现有的VLM与高分辨率图像理解和推理驱动的生成斗争.
- 在跨模式任务中,开源和专有模型之间存在性能差距.
研究的目的:
- 引入Mini-Gemini,这是一个提高VLM性能的框架.
- 为了解决VLMs视觉代币表示和数据质量的局限性.
- 为了弥合开源和专有VLM之间的性能差距.
主要方法:
- 使用了额外的视觉编码器来进行高分辨率的改进,而不会增加令牌数量.
- 构建了一个高质量的数据集,以改善图像理解和推理.
- 集成的迷你双子与各种密集和混合专家 (MoE) 大语言模型 (LLM).
主要成果:
- 在几个基准上取得了领先的零射击表现,超过了私人模型.
- 在MMB基准上达到80.6%的准确率,在TextVQA上达到74.1%.
- 通过更强大的LLM,视觉编码器和数据,证明了持续的改进.
结论:
- 迷你双子有效地增强了VLM用于图像理解,推理和生成.
- 该框架成功地缩小了与先进专有模型的性能差距.
- 迷你双子为推进开源VLM功能提供了可扩展和有效的方法.
相关概念视频
Multi-input and Multi-variable systems
376
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
In the absence of...
376
Higher Mental Functions of the Brain: Language
3.4K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
3.4K


