PSCSE: prompt-based contrastive learning with sample filtering for unsupervised sentence embedding.

Area of Science:

• Natural Language Processing
• Machine Learning
• Artificial Intelligence

Background:

• Unsupervised sentence representation learning is crucial for NLP tasks.
• Contrastive learning methods excel by optimizing embedding space alignment and uniformity.
• Current methods often overlook negative sample selection, leading to biases and false negatives.

Purpose of the Study:

• To introduce a novel method, prompt-based contrastive learning with sample filtering (PSCSE), for unsupervised sentence embedding.
• To address the limitations of random in-batch negative sampling in contrastive learning.
• To enhance the discriminative ability of sentence embedding models.

Main Methods:

• Utilizing synthesized hard negatives generated via "NOT"-style prompts to optimize representation uniformity.
• Employing an auxiliary encoder for a sample filtering approach to mitigate false negatives.
• Evaluating performance on semantic textual similarity datasets.

Main Results:

• PSCSE achieved superior performance compared to dominant methods like SimCSE, E-SimCSE, and PromptBERT.
• The proposed method demonstrated significant improvements in average Spearman's correlation score.
• Effectively addressed issues of false negatives and sampling biases inherent in previous approaches.

Conclusions:

• PSCSE offers a more effective strategy for unsupervised sentence representation learning.
• The integration of hard negative generation and sample filtering enhances model robustness and accuracy.
• This approach represents a significant advancement in contrastive learning for sentence embeddings.