Sequence labeling via reinforcement learning with aggregate labels.

Area of Science:

• Natural Language Processing (NLP)
• Machine Learning (ML)
• Artificial Intelligence (AI)

Background:

• Sequence labeling is crucial for NLP tasks like Named Entity Recognition (NER), Question Answering (QA), and Information Extraction (IE).
• Traditional supervised ML methods for sequence labeling face challenges with training-evaluation objective mismatches and high costs of token-level annotation.
• Existing methods require extensive, granular ground-truth data, which is labor-intensive and expensive to acquire.

Purpose of the Study:

• To introduce a novel reinforcement learning (RL) approach for sequence labeling.
• To address limitations of supervised methods by using aggregate annotations (e.g., entity mention counts) for feedback.
• To reduce annotation costs and variance in sequence labeling tasks.

Main Methods:

• Developed a reinforcement learning framework for sequence labeling.
• Utilized aggregate annotations, specifically counts of entity mentions, to generate training feedback.
• Experimented with various aggregate feedback mechanisms and reward functions, focusing on Named Entity Recognition (NER).

Main Results:

• Demonstrated that sequence labeling can be effectively learned using purely count-based labels.
• Validated the approach's efficacy even when feedback is provided at the sequence level.
• Showcased the potential of count-based labels to significantly lower annotation expenses and improve consistency.

Conclusions:

• The proposed count-based reinforcement learning method offers a viable alternative to traditional supervised sequence labeling.
• This approach substantially simplifies the annotation process compared to token-level boundary determination.
• The findings suggest a promising direction for more efficient and cost-effective NLP model training.