视觉任务的视觉语言模型:一项调查
IEEE transactions on pattern analysis and machine intelligence
|February 26, 2024
概括
视觉语言模型 (VLMs) 通过从庞大的互联网数据中学习,为视觉识别提供了更有效的方法. 本综述探讨了VLM,其方法,以及改善零射击视觉识别的未来方向.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 传统的视觉识别依赖于劳动密集型,特定任务的深度神经网络 (DNN) 训练,使用人群标记的数据.
- 这种模式对于越来越多的视觉识别任务来说是耗时和低效的.
- 新兴的视觉语言模型 (VLMs) 通过利用大规模的图像文本数据来解决这些局限性.
研究的目的:
- 为各种视觉识别任务提供视觉语言模型 (VLM) 的系统审查.
- 分析VLM基础,包括架构,预培训目标和下游应用.
- 分类和评估现有的VLM预培训,转移学习和知识蒸方法.
主要方法:
- 关于视觉识别的VLM研究的综合文献综述.
- VLM预培训策略,转移学习技术和知识蒸方法的分类.
- 使用广泛采用的数据集对审查的VLM方法进行基准分析和分析.
主要成果:
- VLM从网络尺度的图像-文本对中学习丰富的视觉语言相关性.
- 一个单一的VLM可以在各种视觉识别任务中实现零射击预测.
- 本综述对各种VLM方法及其性能进行了分类和分析.
结论:
- 视觉语言模型代表了与视觉识别传统DNN培训相比的显著进步.
- VLMs提供高效和多功能零射击功能,减少对特定任务标签的依赖.
- 未来的研究方向包括解决当前的挑战和探索视觉识别中的新型VLM应用.
更多相关视频
09:27Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language
Published on: October 13, 2018
10.0K
07:12Development of a Gaze-Contingent Display Framework Designed for Perceptual and Oculomotor Research with Simulated Central Vision Loss
Published on: April 11, 2025
344
相关概念视频
Vision
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
Depth Perception and Spatial Vision
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
Visual System
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
Color Vision
Color perception begins in the retina, the light-sensitive layer at the back of the eye. Two main theories explain how colors are seen: the trichromatic theory and the opponent-process theory. The trichromatic theory, proposed by Thomas Young in 1802 and extended by Hermann von Helmholtz in 1852, suggests that color vision is based on three types of cone receptors in the retina. These cones are sensitive to different but overlapping ranges of wavelengths corresponding to red, blue, and green.
