Dual Debiasing for Noisy In-Context Learning for Text Generation.

Area of Science:

• Artificial Intelligence
• Natural Language Processing

Background:

• In-context learning (ICL) depends on high-quality data.
• Current methods for detecting noisy data using perplexity fail when noise levels are high.

Purpose of the Study:

• To address the limitations of existing perplexity-based noise detection methods in ICL.
• To develop a robust metric for identifying sample cleanliness irrespective of corpus noise levels.

Main Methods:

• Re-examined perplexity biases from annotations and LLM domain knowledge.
• Introduced a dual-debiasing framework using synthesized neighbors to correct perplexity.
• Developed a robust Sample Cleanliness Score.

Main Results:

• The proposed Sample Cleanliness Score accurately identifies noisy samples across varying noise ratios.
• The method demonstrates superior noise-detection capabilities compared to existing approaches.
• ICL performance using the proposed method is comparable to using a fully clean dataset.

Conclusions:

• The dual-debiasing framework effectively overcomes perplexity biases in noisy ICL settings.
• The Sample Cleanliness Score provides a reliable measure of data quality.
• This approach offers robust performance even with extremely high noise ratios in demonstration data.