DDSUD: dynamically detecting subsequence uncertainty and diversity for active learning in imbalanced Chinese...

Area of Science:

• Natural Language Processing
• Machine Learning
• Artificial Intelligence

Background:

• Supervised deep learning methods for Chinese sentiment analysis require extensive labeled data, which is costly and time-consuming to acquire.
• Existing active learning methods struggle with imbalanced datasets and efficiently utilizing limited labeled data.

Purpose of the Study:

• To propose Dynamically Detecting Subsequence Uncertainty and Diversity (DDSUD), a Bidirectional Encoder Representations from Transformers (BERT)-based active learning framework.
• To address the challenges of data scarcity and imbalanced datasets in Chinese sentiment structure analysis.
• To improve the efficiency and effectiveness of active learning for sequence labeling tasks.

Main Methods:

• DDSUD integrates subsequence uncertainty detection, diversity-driven sample selection, and dynamic weighting.
• The framework adaptively balances uncertainty and diversity throughout active learning iterations.
• Utilizes Bidirectional Encoder Representations from Transformers (BERT) for feature extraction.

Main Results:

• DDSUD achieves performance comparable to fully supervised methods using only 50% of the labeled data.
• Outperforms state-of-the-art active learning methods with the same amount of labeled data.
• Demonstrates strong adaptability and generalization in low-resource and imbalanced scenarios, improving minority class recognition.

Conclusions:

• DDSUD offers an efficient and effective solution for Chinese sentiment analysis, particularly in low-resource and imbalanced settings.
• The framework's dynamic adjustment of uncertainty and diversity enhances model performance and generalization.
• Reduces the need for large labeled datasets, making advanced sentiment analysis more accessible.