超越静态图像,在多流视觉理解模型中提高输入方差弹性
Amir Hosein Fadaei1, Mohammad-Reza A Dehaqani2,3
1College of Engineering, School of Electrical and Computer Engineering, University of Tehran, Tehran, Iran.
Scientific reports
|July 4, 2024
概括
使用视频和时间特征训练视觉模型,可以提高它们对输入变化的弹性,与传统的基于图像的方法不同. 这种由大脑启发的方法提高了计算机视觉系统的稳定性.
科学领域:
- 计算机视觉 计算机视觉
- 神经科学是一个神经科学.
- 人工智能的人工智能
背景情况:
- 传统的视觉模型主要使用静态图像的空间特征.
- 自然视觉处理连续的时空信息,提供弹性.
- 现有的视频理解模型与图像理解框架的集成有限.
研究的目的:
- 用视频训练的脑启发视觉模型的开发.
- 调查时空特征对模型弹性的影响.
- 为了弥合图像和视频理解模型之间的差距.
主要方法:
- 开发了一种新的脑启发视觉模型架构.
- 使用包含时间特征的视频数据训练模型.
- 对各种输入变化的模型弹性进行评估.
主要成果:
- 在视频上训练的模型表现出比静态图像上训练的模型更强的弹性.
- 包括时间特征在内显著提高了强度.
- 灵感来自大脑的方法显示了对输入变化的增强适应性.
结论:
- 用视频和时间特征进行培训对于强大的视觉模型至关重要.
- 由大脑启发的模型为先进的计算机视觉提供了一个有希望的方向.
- 这项研究强调了模仿自然视觉处理的好处.
相关概念视频
Multi-input and Multi-variable systems
105
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
105
Improving Translational Accuracy
9.9K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
9.9K
Propagation of Uncertainty from Systematic Error
502
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
502
Reducing Line Loss
150
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
150
Propagation of Uncertainty from Random Error
664
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
664
Variability: Analysis
137
Measures of variability are statistical metrics that reveal the dispersion pattern within a dataset. They are pivotal in biostatistics, providing insights into the heterogeneity within health and biological data. Variability signifies the degree to which data points diverge from one another, helping researchers understand the potential range of values and associated uncertainty within the data.
The range is a simple measure of variability, indicating the difference between the highest and...
The range is a simple measure of variability, indicating the difference between the highest and...
137


