Text Data Augmentation for Deep Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Natural Language Processing

Background:

• Deep Learning models in Natural Language Processing (NLP) often face challenges with generalization and overfitting.
• Data Augmentation is a key training strategy with significant potential for advancing NLP.
• NLP's application of Data Augmentation is less mature compared to Computer Vision.

Purpose of the Study:

• To survey the application of Data Augmentation strategies in Natural Language Processing.
• To explore major motifs, frameworks, and practical implementation tools for text data augmentation.
• To identify key differences, promising ideas, and future research directions for Data Augmentation in NLP.

Main Methods:

• Categorization of Data Augmentation motifs: strengthening decision boundaries, brute force training, causality, and meaning vs. form.
• Review of existing text data augmentation frameworks and tools.
• Analysis of studies on using augmentations for generalization and overfitting characterization.

Main Results:

• Data Augmentation offers methods to construct test sets for evaluating model generalization.
• Identified practical tools include consistency regularization, controllers, and augmentation pipelines (offline/online).
• Highlighted promising research areas: task-specific augmentations, self-supervised learning integration, transfer learning, and AI-Generating Algorithms.

Conclusions:

• Data Augmentation is a crucial yet underexplored area for enhancing NLP model performance and reliability.
• Further research into advanced augmentation techniques and their integration with other learning paradigms is warranted.
• This survey aims to stimulate further interest and research in Text Data Augmentation.