基于Vision-Pro的自我中心空间图像的感知质量评估
IEEE transactions on visualization and computer graphics
|March 11, 2025
概括
研究人员开发了一个新的数据库和模型,用于评估扩展现实 (XR) 中的自我中心空间图像的质量. 该ESIQAnet模型在预测各种显示模式的感知质量方面表现出卓越的性能.
科学领域:
- 计算机视觉 计算机视觉
- 虚拟现实 虚拟现实 虚拟现实
- 图像处理 图像处理
背景情况:
- 扩展现实 (XR) 和头戴显示器 (HMD) 正在发展,以自我中心的空间图像成为关键的内容类型.
- 对XR内容的体验质量 (QoE) 的评估至关重要,但以自我为中心的空间图像对传统的图像质量评估 (IQA) 提出了独特的挑战.
- 现有的IQA研究还没有充分解决自我中心空间图像的特定特征.
研究的目的:
- 建立第一个专门用于自我中心空间图像的图像质量评估 (IQA) 数据库.
- 提出一种新的深度学习模型,用于预测自我中心空间图像的感知质量.
- 为了评估模型在XR相关的不同显示模式中的性能.
主要方法:
- 创建了以自我为中心的空间图像质量评估数据库 (ESIQAD),其中包括500张图像和2D,3D窗口和3D沉浸式显示的平均意见得分 (MOSs).
- 开发ESIQAnet,一个基于mamba2的多阶段特征融合模型,利用视觉状态空间二元化 (VSSD) 块,交叉注意力和转移注意力.
- 特征提取,双筒视图信息的融合,特征精细化和质量回归用于感知质量预测.
主要成果:
- 与22个最先进的IQA模型相比,ESIQAnet模型表现出更高的性能.
- 该模型在所有三种测试显示模式 (2D,3D窗口,3D沉浸式) 中预测知觉质量的高准确度.
- 建立的ESIQAD为未来对自我中心空间图像质量研究提供了宝贵的资源.
结论:
- 拟议的ESIQAnet模型有效地预测了XR中自我中心空间图像的感知质量.
- ESIQAD对该领域做出了重大贡献,使其能够进一步开发和对沉浸式内容的IQA方法进行基准测试.
- 这项工作解决了IQA研究中对于新兴XR视觉媒体的关键差距.
更多相关视频
相关概念视频
Depth Perception and Spatial Vision
503
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
503
Vision
52.9K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
52.9K
Perceptual Constancy
315
Perceptual constancy is the ability to recognize that objects remain consistent and unchanged even when their appearance varies due to changes in sensory input. There are four main types of perceptual constancy: size constancy, shape constancy, color constancy, and brightness constancy.
Size constancy is the recognition that an object remains the same size, even when its image on the retina changes. For instance, a bus is perceived to be large enough to carry people, even if it looks tiny from...
Size constancy is the recognition that an object remains the same size, even when its image on the retina changes. For instance, a bus is perceived to be large enough to carry people, even if it looks tiny from...
315


