Learning to Forget for Meta-Learning via Task-and-Layer-Wise Attenuation.

Area of Science:

• Artificial Intelligence
• Machine Learning

Background:

• Few-shot learning aims for generalization from limited data.
• Meta-learning leverages prior knowledge across tasks to accelerate learning.
• Model-Agnostic Meta-Learning (MAML) uses a shared initialization for rapid adaptation.

Purpose of the Study:

• To address task conflicts and compromised optimization landscapes caused by shared initializations in MAML.
• To propose a method that dynamically adjusts the influence of prior knowledge based on task and layer specifics.

Main Methods:

• Introduced task-and-layer-wise attenuation on the compromised initialization.
• Developed the 'Learn to Forget' (L2F) method to selectively forget detrimental prior knowledge.
• Applied L2F to MAML-based frameworks.

Main Results:

• L2F significantly enhances the performance of MAML across various domains.
• Demonstrated improvements in few-shot classification, cross-domain few-shot classification, regression, reinforcement learning, and visual tracking.
• Showcased the effectiveness of dynamic, layer-specific attenuation in mitigating initialization conflicts.

Conclusions:

• Task-and-layer-wise attenuation is crucial for overcoming MAML's limitations.
• L2F offers a robust solution for improving meta-learning adaptation and generalization.
• The proposed method generalizes well across a wide spectrum of machine learning tasks.