Enhancing Text Datasets With Scaling and Targeting Data Augmentation to Improve BERT-Based Machine Learners

Area of Science:

• Machine Learning
• Natural Language Processing
• Medical Informatics

Background:

• Acquiring sufficient data for machine learning is challenging, especially for text data.
• Large language models (LLMs) offer solutions for synthetic text data generation.
• Autism spectrum disorder (ASD) diagnosis can benefit from improved machine learning models.

Purpose of the Study:

• To analyze the impact of descriptively-selected synthetic data on downstream classifier performance for ASD.
• To compare different synthetic data augmentation schemes (Data Targeting vs. Data Scaling).
• To evaluate the effectiveness of white-box metrics in guiding data selection.

Main Methods:

• A finetuned multilabel, bidirectional encoder model was used to label 10,892 behavioral descriptions with seven ASD diagnostic criteria.
• Synthetic data augmentation was applied at 50% and 100% of the baseline dataset size.
• Data points were selected using type-token ratio, cosine similarity, and perplexity metrics.
• Performance was evaluated using precision, recall, and F1 scores per label.

Main Results:

• Synthetic data augmentation consistently increased recall by approximately 8% but decreased precision by approximately 10%.
• White-box metrics and stability analysis did not show a clear relationship with the observed performance changes.
• Data Targeting augmentation demonstrated potential cost reduction for the BioBERT model.

Conclusions:

• Synthetic data augmentation impacts classifier performance, with varying effects on precision and recall.
• The choice of augmentation scheme should align with the specific application, such as medical screening or diagnosis.
• Further research is needed to refine synthetic data selection methods for optimal performance in clinical settings.