在全向视频中,音频如何影响视觉注意力? 数据库和模型
概括
这项研究引入了一个全方位视频 (ODV) 的新视听突出数据库和一个新的预测网络 (OmniAVS). 研究结果表明,音频显著影响ODV的视觉注意力,提高了预测准确度.
科学领域:
- 计算机视觉 计算机视觉
- 人与计算机的交互
- 多媒体系统 多媒体系统
背景情况:
- 在全向视频 (ODV) 中,观众的注意力对于虚拟和增强现实参与至关重要.
- 由于缺乏大规模的数据集和分析,对ODV中视听突出性预测的研究有限.
- 现有的模型往往无法有效地整合ODV突出性的音频和视觉线索.
研究的目的:
- 从主观和客观的角度研究ODV中的视听注意力.
- 为ODV引入一种新的视听突出数据库 (AVS-ODV),并分析音频对视觉注意力的影响.
- 为ODVs开发一个先进的视听突出性预测模型.
主要方法:
- 收集了来自60名受试者在162个ODV中在不同音频条件下 (无声,单声,双声) 的眼睛跟踪数据,以创建AVS-ODV数据库.
- 通过使用新数据库,分析了音频对ODV视觉注意力的影响.
- 开发和评估了基于U-Net的模型OmniAVS网络,用于层次化的视听功能融合.
主要成果:
- AVS-ODV数据库提供了一个全面的资源,用于研究ODV中的视听注意力.
- 音频显著影响ODV的视觉注意力模式.
- 拟议的OmniAVS模型在ODV和传统的AVS预测任务中,与最先进的模型相比,实现了更高的性能.
结论:
- AVS-ODV数据库和OmniAVS模型代表了对ODV的视听突出预测的重大进展.
- 音频和视觉信息的有效融合是沉浸式环境中准确预测注意力的关键.
- 这项研究为更具吸引力和现实的虚拟和增强现实体验铺平了道路.
相关概念视频
Auditory Perception
288
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the...
288
Perceiving Loudness, Pitch, and Location
167
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
167
Depth Perception and Spatial Vision
483
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
483
Perception of Sound Waves
4.4K
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
4.4K
Factors Affecting Perception
1.4K
Perception is influenced by perceptual set, context, motivation, and emotion. Perceptual set, or perceptual expectancy, refers to the tendency to perceive things in a particular way, influenced by previous experiences and expectations. This phenomenon affects the interpretation of stimuli, creating a set of mental tendencies and assumptions that impact sensory perceptions of sound, taste, touch, and sight.
An illustrative example of a perceptual set is the scenario where an airline pilot told...
An illustrative example of a perceptual set is the scenario where an airline pilot told...
1.4K


