Hierarchical Knowledge Propagation and Distillation for Few-Shot Learning.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computer Vision

Background:

• Few-Shot Learning (FSL) research has advanced, but often focuses on the transductive setting, limited by small unlabeled query sets.
• Existing inductive FSL methods primarily optimize feature extraction, potentially overlooking crucial sample-class relationship dynamics in low-data regimes.

Purpose of the Study:

• To propose an inductive Few-Shot Learning (FSL) framework, Hierarchical Knowledge Propagation and Distillation (HKPD), designed to enhance representation learning.
• To address the limitations of existing FSL methods by explicitly modeling both sample-level and class-level relationships.

Main Methods:

• HKPD constructs a sample-level information propagation module to explore pairwise sample relationships for discriminative representations.
• A class-level information propagation module is designed to effectively obtain and update class-level information.
• A self-distillation module integrates hierarchical knowledge, propagating information across modules to refine learned representations.

Main Results:

• The proposed HKPD method demonstrates superior performance on standard few-shot benchmark datasets.
• Experiments confirm the effectiveness of the hierarchical knowledge propagation and distillation approach in improving FSL performance.
• HKPD surpasses current state-of-the-art methods in few-shot classification tasks.

Conclusions:

• HKPD offers a novel inductive approach to Few-Shot Learning by effectively leveraging hierarchical knowledge.
• The framework's ability to model inter-sample and sample-class relationships is key to its improved performance in low-data scenarios.
• This work advances inductive FSL by providing a robust method for learning from limited labeled data.