A dimensional structure based knowledge distillation method for cross-modal learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Cross-modal learning (CML) often relies on transferring knowledge from high-quality to low-quality data modalities.
• The underlying reasons for the effectiveness of knowledge transfer in CML are not well understood.

Purpose of the Study:

• To investigate the relationship between feature discriminability and dimensional structure (DS) in CML.
• To propose a novel DS-based knowledge distillation method to enhance CML performance, particularly when data quality differs significantly.

Main Methods:

• Analyzing the correlation between feature discriminability and DS by examining features from high- and low-quality modalities.
• Expressing DS using spatial distribution of intermediate features and channel-wise correlation of output features.
• Developing a DS-based knowledge distillation technique that enforces uniform intermediate feature distribution and channel-wise independence in the low-quality modality.

Main Results:

• High-quality features exhibit superior DS compared to low-quality features.
• The proposed DS-based knowledge distillation method significantly improves CML performance.
• Introduced the IIS-ISCAS dataset for marine target recognition, featuring paired optics and radar modalities.

Conclusions:

• The dimensional structure of features is crucial for discriminability and effective knowledge transfer in CML.
• DS-based knowledge distillation offers a promising approach to enhance CML, especially in scenarios with imbalanced data quality.
• The IIS-ISCAS dataset will facilitate further research in CML for marine target recognition.