Effective Transfer Learning with Label-Based Discriminative Feature Learning.

Area of Science:

• Natural Language Processing
• Machine Learning
• Artificial Intelligence

Background:

• Transfer learning with pre-trained language models improves NLP task performance.
• Pre-training on general data can lead to learning irrelevant features for specific downstream tasks.

Purpose of the Study:

• To propose a novel learning method for embedding pre-trained models to learn task-specific features.
• To address the issue of pre-trained models learning general rather than specific features.

Main Methods:

• A novel learning method is proposed using label embedding and sampled data pairs.
• Contrast learning is employed to learn label features of downstream tasks.
• Experiments were conducted on sentence classification datasets.

Main Results:

• The proposed method enables pre-trained models to learn features specific to downstream tasks.
• PCA and clustering of embeddings were used to evaluate feature learning.
• Effectiveness demonstrated on sentence classification tasks.

Conclusions:

• The proposed method enhances transfer learning by focusing on task-specific feature acquisition.
• This approach improves the relevance of pre-trained models for specialized NLP applications.