DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learning.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Meta-learning is crucial for advancing machine learning but requires extensive, high-quality meta-data.
• Existing meta-data-driven approaches struggle with imperfect or limited training data.
• Acquiring high-quality meta-data for real-world applications can be challenging and costly.

Purpose of the Study:

• To introduce a novel meta-knowledge informed meta-learning (MKIML) framework.
• To enhance meta-learning performance by integrating compensated meta-knowledge.
• To address the limitations of current meta-learning methods when dealing with imperfect training tasks.

Main Methods:

• Proposed a meta-knowledge informed meta-learning (MKIML) framework.
• Integrated task-agnostic meta-knowledge into the meta-objective using meta-regularization (MR).
• Introduced data augmentation consistency (DAC) to encode invariance as meta-knowledge, forming DAC-MR for the MR objective.

Main Results:

• DAC-MR enables learning well-performing meta-models from noisy, sparse, or unavailable meta-data.
• Theoretically demonstrated DAC-MR as a proxy meta-objective for meta-model evaluation without high-quality meta-data.
• DAC-MR combined with meta-data-driven loss achieved superior meta-level generalization across 12 diverse meta-learning tasks.

Conclusions:

• DAC-MR effectively aids meta-model learning, demonstrating robust performance across various settings.
• The proposed method is problem-agnostic and shows potential for broad application in meta-learning.
• Experimental results align with theoretical insights, validating the efficacy of DAC-MR.