在事件驱动感知中,通过连续流评估框架弥合延迟差距
Jie Chu1,2, Runze Zhang1, Chu Yang3
1Generalist Embodied Intelligence Lab, School of Computer Science and Technology, Tongji University, Shanghai, China.
Nature communications
|March 17, 2026
概括
神经形态视觉系统由于延迟而面临评估挑战. 新的基于STream的 lAtency-awaRe Evaluation (STARE) 框架量化了这种影响,改善了现实世界的表现.
科学领域:
- 计算机视觉 计算机视觉
- 神经形态工程的神经形态工程
背景情况:
- 神经形态视觉系统为实时应用程序处理事件流.
- 目前的评估方法使用RGB成像,无视感知延迟,造成性能差距.
研究的目的:
- 引入一个新的评估框架,基于STream的 lAtency-awaRe评估 (STARE),以解决神经形态视觉中的延迟问题.
- 量化延迟对在线准确性的影响,并提出解决方案.
主要方法:
- 开发了STARE框架,包括持续采样和延迟意识评估.
- 创建ESOT500,一个高动态的对象跟踪数据集,有500Hz的注释用于验证.
- 引入了异步跟踪和上下文感知采样,用于模型增强.
主要成果:
- 发现延迟会使在线准确性降低50%以上.
- 在STARE框架有效量化延迟诱导的准确性损失.
- 拟议的增强策略改善了模型吞吐量和处理低事件密度的情况.
结论:
- STARE框架弥合了神经形态视觉系统的理论潜力和现实世界的部署之间的差距.
- 解决延迟对于准确可靠的实时性能至关重要.
- 新的评估和增强方法为更强大的神经形态应用铺平了道路.
更多相关视频
07:12Development of a Gaze-Contingent Display Framework Designed for Perceptual and Oculomotor Research with Simulated Central Vision Loss
Published on: April 11, 2025
1.0K
09:27Author Spotlight: Exploring the Link Between Time Perception of Visual Stimuli and Reading Skills
Published on: January 19, 2024
1.8K
相关概念视频
Perception
1.6K
Perception is a fundamental psychological process that enables individuals to organize, interpret, and consciously experience sensory information. This process is crucial for understanding and interacting with the world around us. It includes both bottom-up and top-down processing, each playing a distinct role in how we perceive our environment.
Bottom-up processing begins at the sensory level, where receptors detect external environmental stimuli. These could include the tactile sensation of...
Bottom-up processing begins at the sensory level, where receptors detect external environmental stimuli. These could include the tactile sensation of...
1.6K
Parallel Processing
858
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
858
Sampling Continuous Time Signal
816
In signal processing, a continuous-time signal can be sampled using an impulse-train sampling technique, followed by the zero-order hold method. Impulse-train sampling involves the use of a periodic impulse train, which consists of a series of delta functions spaced at regular intervals determined by the sampling period. When a continuous-time signal is multiplied by this impulse train, it generates impulses with amplitudes corresponding to the signal's values at the sampling points.
In the...
In the...
816
Perception of Sound Waves
5.9K
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
5.9K
Introducing Social Perception
522
Perceiving others accurately is fundamental to effective communication and relationship-building. Social perception, a key concept in social psychology, refers to the cognitive processes through which individuals gather and interpret information about others to understand their actions, intentions, and motivations. This process extends beyond spoken words and overt behaviors, incorporating subtle nonverbal cues and contextual factors.Nonverbal Cues and Their SignificanceNonverbal cues play a...
522
Auditory Perception
1.4K
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the...
1.4K
