3D Affine: An Embedding of Local Image Features for Viewpoint Invariance Using RGB-D Sensor Data

Area of Science:

• Computer Vision
• Robotics
• 3D Reconstruction

Background:

• Local image features are crucial for computer vision but struggle with significant viewpoint changes.
• Existing methods fail beyond 30° out-of-plane rotations, limiting applications like wide baseline matching and 6D pose estimation.

Purpose of the Study:

• To develop a general embedding method that enhances viewpoint invariance of local image features.
• To exploit depth information from RGB-D images for more robust feature detection and description.

Main Methods:

• Proposed a general embedding that wraps existing local image feature detectors/descriptors.
• Utilized depth maps to locate smooth surfaces and project them into a viewpoint-invariant representation.
• Evaluated performance using synthetic and real-world objects with varying geometries.

Main Results:

• The proposed embedding significantly boosted viewpoint invariance, achieving an average of 45.4° across datasets.
• Features from objects with surface discontinuities reached an average invariance of 52.8°.
• 19 out of 20 local features, when embedded, exceeded 60° viewpoint difference, compared to only one standalone feature.

Conclusions:

• The novel embedding method substantially increases viewpoint invariance for local image features.
• The approach is effective even with depth noise from low-cost sensors.
• This technique enables more robust 3D computer vision applications requiring large viewpoint tolerance.