6D Object Pose Estimation Based on Cross-Modality Feature Fusion.

Area of Science:

• Robotics
• Computer Vision
• Artificial Intelligence

Background:

• Accurate 6D pose estimation is crucial for robotic applications.
• Current methods often concatenate RGB and depth data, neglecting inter-modal interactions, leading to reduced accuracy in challenging conditions like occlusion and illumination changes.

Purpose of the Study:

• To develop an advanced method for fusing RGB and depth features for improved 6D pose estimation.
• To enhance the integration of individual modality information and cross-modality interactions.

Main Methods:

• Depth images are converted to point clouds and processed using PointNet++.
• CNNs and attention mechanisms extract RGB features, capturing intra-modality context.
• A Cross-Modality Feature Fusion Module (CFFM) and a Feature Contribution Weight Training Module (CWTM) are proposed for effective feature fusion and modality contribution allocation.

Main Results:

• The proposed method achieves high accuracy on benchmark datasets.
• On the LineMOD dataset, an accuracy of 96.9% (ADD(-S) metric) was reached.
• On the YCB-Video dataset, 94.7% (ADD-S AUC) and 96.5% (ADD-S score <2 cm) accuracy were obtained.

Conclusions:

• The novel fusion strategy maximizes inter- and intra-modality information integration.
• Considering modality contributions enhances overall model robustness, outperforming existing methods in challenging scenarios.