Source-Free Cross-Modal Knowledge Transfer by Unleashing the Potential of Task-Irrelevant Data.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Source-free cross-modal knowledge transfer is challenging due to memory and privacy constraints preventing access to task-relevant source data.
• Existing methods use paired task-irrelevant data to bridge modality gaps but overlook its potential for estimating source data distribution.

Purpose of the Study:

• To propose a novel framework for enhancing source-free cross-modal knowledge transfer by effectively utilizing paired task-irrelevant data.
• To improve the estimation of source data distribution and facilitate more effective knowledge transfer to the target modality.

Main Methods:

• Introduced a Task-irrelevant data-Guided Modality Bridging (TGMB) module to translate target modality data into source-like images, addressing inter- and intra-modality gaps.
• Developed a Task-irrelevant data-Guided Knowledge Transfer (TGKT) module leveraging paired task-irrelevant data.
• Incorporated a self-supervised pseudo-labeling approach within TGKT to enable target model learning from its own predictions due to unlabeled target data.

Main Results:

• The proposed framework achieved state-of-the-art performance on RGB-to-depth and RGB-to-infrared transfer tasks.
• Demonstrated the effectiveness of the TGMB and TGKT modules in bridging modality gaps and facilitating knowledge transfer.
• Validated the benefits of the self-supervised pseudo-labeling approach for learning with unlabeled target data.

Conclusions:

• The novel framework effectively unlocks the potential of paired task-irrelevant data for source-free cross-modal knowledge transfer.
• The proposed approach offers a significant advancement in handling modality gaps and data constraints in cross-modal learning.
• The method provides a robust solution for real-world applications where source data access is limited.