Comprehensive Visual Question Answering on Point Clouds through Compositional Scene Manipulation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• 3D Scene Understanding

Background:

• Visual Question Answering on 3D Point Clouds (VQA-3D) is an emerging challenge.
• Existing datasets may lack diversity and real-world complexity for robust VQA-3D models.

Purpose of the Study:

• To introduce CLEVR3D, a large-scale dataset for VQA-3D.
• To facilitate comprehensive visual understanding in 3D scenes by addressing confounding biases and common-sense context.
• To improve the performance of VQA-3D models on real-world comprehension tasks.

Main Methods:

• Developed a question engine using 3D scene graphs to generate diverse reasoning questions on object attributes and spatial relationships.
• Created an initial set of 44K questions from 1,333 real-world 3D scenes.
• Introduced compositional scene manipulation to generate 127K questions from 7,438 augmented 3D scenes, addressing confounding biases and common-sense context.

Main Results:

• The CLEVR3D dataset comprises 171K questions across 8,771 3D scenes.
• The proposed challenging setup and augmented scenes improve VQA-3D model comprehension.
• Baselines demonstrate that CLEVR3D significantly boosts performance on 3D scene understanding tasks.

Conclusions:

• CLEVR3D is a valuable resource for advancing VQA-3D research.
• The dataset's diverse and challenging nature pushes the boundaries of 3D visual understanding.
• Future VQA-3D models trained on CLEVR3D are expected to exhibit enhanced real-world applicability.