From Channel Bias to Feature Redundancy: Uncovering the "Less is More" Principle in Few-Shot Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Deep neural networks (DNNs) face challenges in adapting learned representations to novel tasks, particularly under distribution shifts and with limited data.
• A key obstacle is channel bias, where DNNs maintain an overemphasis on source-task features that are misaligned with the requirements of new tasks.

Purpose of the Study:

• To identify and address the core reasons behind DNNs' failure to adapt representations in few-shot learning scenarios.
• To propose a novel method for improving representation transfer and few-shot learning robustness.

Main Methods:

• Theoretical analysis to understand the origin of feature redundancy in few-shot learning.
• Development and application of Augmented Feature Importance Adjustment (AFIA), a soft-masking technique to estimate feature importance from augmented data.

Main Results:

• Demonstrated that a small fraction (1-5%) of discriminative features significantly improves few-shot classification accuracy, indicating widespread feature redundancy.
• Confirmed that confounding feature dimensions (high intra-class variance, low inter-class separability) contribute to redundancy, especially in low-data regimes.
• Showcased the effectiveness of AFIA in mitigating feature redundancy and enhancing few-shot learning performance.

Conclusions:

• Established a direct link between channel bias and extreme feature redundancy in few-shot learning.
• Proposed AFIA as a practical and effective method for improving few-shot representation transfer.
• Highlighted the "Less is More" phenomenon as a critical characteristic of few-shot learning, emphasizing the need for feature selection and regularization.