Dynamic Augmentation Data Selection for Few-shot Text Classification.

Area of Science:

• Natural Language Processing
• Machine Learning

Background:

• Data augmentation is crucial for enhancing pre-trained language model (PLM) performance and robustness.
• Existing methods often rely on in-sample or out-of-sample augmentation, with data quality being a critical factor.

Purpose of the Study:

• To propose a dynamic data selection method for identifying effective augmentation data tailored to a model's learning stage.
• To improve the fine-tuning process by selecting augmentation samples that optimally facilitate current model learning.

Main Methods:

• A curriculum learning strategy is employed to filter out noisy pseudo-labeled augmentation samples.
• Augmentation data effectiveness is estimated using influence scores on the model at each update, enabling parameter-specific selection.
• A two-stage augmentation strategy integrates in-sample and out-of-sample data across different learning phases.

Main Results:

• The proposed dynamic data selection method significantly outperforms strong baselines across various sentence classification tasks.
• Experiments demonstrate the effectiveness of selecting augmentation data dynamically based on the model's learning progression.
• Analysis confirms the dynamic nature of data effectiveness and the importance of model learning stages.

Conclusions:

• The dynamic data selection method offers a superior approach to utilizing augmentation data for PLM fine-tuning.
• Tailoring data selection to model learning stages is essential for maximizing the benefits of data augmentation.
• This approach enhances model robustness and performance through optimized data utilization.