综合视觉问题在点云上通过组合场景操纵来回答问题
IEEE transactions on visualization and computer graphics
|December 8, 2023
概括
我们介绍了CLEVR3D,这是一个用于3D点云 (VQA-3D) 视觉问题答案的大数据集. 这一数据集通过产生关于对象属性和空间关系的各种问题来增强3D场景的理解.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 3D场景理解 3D场景理解
背景情况:
- 在3D点云 (VQA-3D) 上进行视觉问题答案是一个新兴的挑战.
- 现有的数据集可能缺乏多样性和现实世界的复杂性,对于强大的VQA-3D模型来说.
研究的目的:
- 介绍CLEVR3D,这是一个大规模的VQA-3D数据集.
- 通过解决混偏见和常识上下文,促进3D场景的全面视觉理解.
- 提高VQA-3D模型在现实世界理解任务中的性能.
主要方法:
- 开发了一个使用3D场景图形生成对象属性和空间关系的各种推理问题的问题引擎.
- 从1333个真实世界3D场景中创建了最初一组44K个问题.
- 引入了组合场景操纵,从7,438个增强3D场景中生成127K个问题,解决混偏见和常识上下文.
主要成果:
- 该CLEVR3D数据集包括171K个问题,涉及8771个3D场景.
- 提出的具有挑战性的设置和增强场景改善了VQA-3D模型的理解.
- 基线表明,CLEVR3D显著提高了3D场景理解任务的性能.
结论:
- CLEVR3D是推动VQA-3D研究的宝贵资源.
- 数据集的多样性和挑战性推动了3D视觉理解的界限.
- 预计未来在CLEVR3D上训练的VQA-3D模型将显示出增强的现实应用性.
相关概念视频
Depth Perception and Spatial Vision
668
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
668
Visual System
585
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
585
Collisions in Multiple Dimensions: Problem Solving
4.2K
In multiple dimensions, the conservation of momentum applies in each direction independently. Hence, to solve collisions in multiple dimensions, we should write down the momentum conservation in each direction separately. To help understand collisions in multiple dimensions, consider an example.
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
4.2K
Vision
53.5K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
53.5K
Relative Motion Analysis using Rotating Axes-Problem Solving
406
Consider a crane whose telescopic boom rotates with an angular velocity of 0.04 rad/s and angular acceleration of 0.02 rad/s2. Along with the rotation, the boom also extends linearly with a uniform speed of 5 m/s. The extension of the boom is measured at point D, which is measured with respect to the fixed point C on the other end of the boom. For the given instant, the distance between points C and D is 60 meters.
Here, in order to determine the magnitude of velocity and acceleration for point...
Here, in order to determine the magnitude of velocity and acceleration for point...
406
Three-Dimensional Force System:Problem Solving
669
A three-dimensional force system refers to a scenario in which three forces act simultaneously in three different directions. This type of problem is commonly encountered in physics and engineering, where it is necessary to calculate the resultant force on the system, which can then be used to predict or analyze the behavior of the object or structure under consideration.
To solve a three-dimensional force system, first resolve each force into its respective scalar components. Do this using...
To solve a three-dimensional force system, first resolve each force into its respective scalar components. Do this using...
669


