冒险家:优化视野 马巴建筑设计以提高效率
Feng Wang1, Timing Yang1, Yaodong Yu2
1Johns Hopkins University.
概括
冒险家模型将图像视为序列,使用单向语言模型进行视觉表示. 这种方法为高分辨率图像处理提供了高效准确的权衡.
科学领域:
- 计算机视觉 计算机视觉
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 高分辨率和细粒度图像对现有模型构成重大计算和内存挑战.
- 目前的视觉表示学习方法由于二次复杂性,往往难以扩展.
研究的目的:
- 介绍Adventurer系列模型,以实现高效的视觉表示学习.
- 解决处理高分辨率图像的计算和内存限制.
主要方法:
- 将图像视为补丁令牌的序列.
- 采用单向的语言模型来学习视觉表现.
- 使用全球聚合令牌和翻转操作,以无地集成到因果推理框架中.
主要成果:
- 与Deit和Vim相比,冒险者模型实现了最佳的效率-精度权衡.
- 冒险者基础在ImageNet-1k上获得了84.3%的测试准确性,其训练吞吐量为216张图像/秒.
- 与Vim和Deit相比,演示了3.8倍和6.2倍更快的训练吞吐量.
结论:
- 冒险者架构提供了显著的计算和内存效率.
- 线性复杂性允许高分辨率和细粒度图像的有效缩放.
- 对复杂的视觉数据的长序列建模的未来研究有潜力.
相关概念视频
Vision
59.3K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
59.3K
Visual System
1.6K
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
1.6K
Depth Perception and Spatial Vision
1.8K
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
1.8K

