Learning 3D Object Templates by Quantizing Geometry and Appearance Spaces.

Area of Science:

• Computer Vision
• Machine Learning
• 3D Shape Representation

Background:

• Traditional 3D object-centered models offer advantages over 2D models but face challenges in learning and inference due to geometric complexity and the 2D-3D appearance gap.
• Existing literature has under-explored 3D model learning despite their potential for lower complexity and better view generalization.

Purpose of the Study:

• To address the complexities of 3D shape representation and the appearance gap in 3D model learning.
• To introduce and evaluate a novel And-Or Tree (AoT) representation for efficient learning of 3D object templates.

Main Methods:

• Developed an And-Or Tree (AoT) representation with geometry-AoT and appearance-AoT components to quantize 3D shape and appearance spaces.
• Transformed 3D object template learning into an efficient structure search problem solvable via dynamic programming.
• Collected a new diverse car dataset and focused on learning 3D car templates integrating shape and appearance models.

Main Results:

• The AoT representation demonstrates superior space efficiency compared to the octree method.
• Learned 3D car templates achieve state-of-the-art results in car detection and pose estimation on a public dataset.
• On a new dataset, the AoT model successfully performs joint object detection, pose/view estimation, and part localization, outperforming the DPM v5 model.

Conclusions:

• The And-Or Tree (AoT) representation effectively tackles the challenges in learning 3D object models from images.
• The proposed method integrates shape and appearance information, enabling robust performance across diverse views and tasks.
• This approach offers a promising direction for advancing 3D object understanding in computer vision.