从人类的注意力中学习属性辅助视觉识别
IEEE transactions on pattern analysis and machine intelligence
|September 11, 2024
概括
这项研究介绍了一种属性注意网络 (A2Net),该网络从人类目光数据中学习,以改进零射击学习 (ZSL) 和细粒度视觉分类 (FGVC). 通过将人工智能注意力与人类注意力对齐,该模型提高了对象识别精度.
科学领域:
- 计算机视觉 计算机视觉
- 机器学习 机器学习
- 认知科学 认知科学
背景情况:
- 人类对象识别依赖于局部属性,这对于零射击学习 (ZSL) 和细粒度视觉分类 (FGVC) 至关重要.
- 神经网络中的注意力机制学习了歧视性属性,但往往忽视了与人类注意力的定位和对齐.
- 现有的方法专注于区域嵌入,忽视了精确属性本地化的重要性.
研究的目的:
- 通过将真实的人类目光数据集成到神经网络中,开发一种新的视觉识别方法.
- 为ZSL和FGVC任务提出一个统一的属性注意网络 (A2Net),它可以从人类的注意力中学习.
- 调查人工智能模型中学习的注意力是否真正模仿人类的视觉注意力.
主要方法:
- 设计了一个统一的属性注意网络 (A2Net),具有属性注意分支和基线分类网络.
- 利用属性原型生成属性注意力图和特征,将它们与人类目光数据对齐.
- 采集真实的人类目光数据使用眼球追踪器在鸟类分类游戏中使用CUB数据集.
- 对齐提取的属性特征与属性定义的类嵌入进行增强的学习.
主要成果:
- 当A2Net模型与人类目光数据进行训练时,它在ZSL和FGVC任务中表现出更好的准确性.
- 实验验证实了从人类注意力学习到视觉识别的有效性.
- 该研究证实了收集人工智能模型开发的人类目光数据集的好处.
结论:
- 整合真实的人类目光数据显著提高了视觉识别模型的性能,如A2Net.
- 拟议的A2Net有效地从人类的注意力中学习,改善了属性本地化和识别.
- 这项研究突出了人类凝视数据和凝视估计算法的价值,用于推进高级计算机视觉任务.
相关概念视频
01:21Association Areas of the Cortex
5.2K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
5.2K
01:26Visual System
557
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
557
00:56Attribution Theory
12.9K
Behavior is a product of both the situation (e.g., cultural influences, social roles, and the presence of bystanders) and of the person (e.g., personality characteristics). Subfields of psychology tend to focus on one influence or behavior over others. Situationism is the view that our behavior and actions are determined by our immediate environment and surroundings. In contrast, dispositionism holds that our behavior is determined by internal factors (Heider, 1958).
12.9K
01:24Vision
53.1K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
53.1K