Visual-Depth Matching Network: Deep RGB-D Domain Adaptation With Unequal Categories

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Traditional domain adaptation (DA) assumes identical label spaces and single-source data, limiting real-world applicability.
• Practical scenarios involve differing label sets and multi-modal data, posing significant challenges for existing DA methods.
• Recognizing RGB images using RGB-D data under label space inequality is a complex, practical problem.

Purpose of the Study:

• To develop a novel deep learning model capable of addressing domain adaptation with label space inequality.
• To effectively utilize multi-modal data (RGB and depth) from source domains for target domain recognition.
• To overcome challenges posed by domain mismatch and differing category distributions.

Main Methods:

• Proposed the Visual-Depth Matching Network (VDMN), a deep model trained end-to-end.
• Incorporated two novel modules and a matching component within VDMN.
• VDMN is designed to jointly identify common and outlier categories, leveraging depth information.

Main Results:

• VDMN demonstrates superior performance on various domain adaptation datasets.
• The model effectively handles domain distribution mismatch under label inequality.
• Significant improvements were observed, particularly in scenarios with unequal label spaces.

Conclusions:

• VDMN offers a robust solution for domain adaptation with label space inequality.
• The model successfully integrates depth information to enhance RGB image recognition.
• VDMN surpasses state-of-the-art methods, especially in challenging, real-world DA scenarios.