3DCoMPaT⁺⁺: An Improved Large-Scale 3D Vision Dataset for Compositional Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• 3D Data Analysis

Background:

• The field of 3D vision requires comprehensive datasets for advancing multimodal and compositional learning.
• Existing datasets often lack the scale, detail, or compositional complexity needed for advanced 3D understanding tasks.

Purpose of the Study:

• To introduce 3DCOMPAT++, a large-scale multimodal 2D/3D dataset designed to facilitate research in compositional 3D vision.
• To establish a new benchmark task, Grounded CoMPaT Recognition (GCR), for recognizing and grounding material compositions on 3D object parts.

Main Methods:

• Generation of 160 million rendered views from over 10 million stylized 3D shapes with part-instance level annotations.
• Inclusion of diverse data modalities: RGB point clouds, 3D textured meshes, depth maps, and segmentation masks.
• Development and evaluation of methods for the novel GCR task, including a modified PointNet++ model.

Main Results:

• The 3DCOMPAT++ dataset encompasses 42 shape categories, 275 part categories, and 293 material classes.
• The GCR task was explored through a data challenge at CVPR, highlighting effective approaches.
• Public release of the dataset and code to support future research.

Conclusions:

• 3DCOMPAT++ provides a valuable resource for advancing multimodal and compositional learning in 3D vision.
• The GCR task and dataset are expected to spur innovation in understanding complex 3D object properties and their compositions.
• The work aims to lower the barrier for future research in compositional 3D vision.