Revisiting Relation Extraction in the era of Large Language Models

Area of Science:

• Natural Language Processing (NLP)
• Artificial Intelligence (AI)

Background:

• Relation Extraction (RE) is a key NLP task for identifying semantic relationships between entities in text.
• Traditional RE methods involve supervised learning for entity and relation identification.
• Recent advancements explore sequence-to-sequence models for RE, generating relations as text.

Purpose of the Study:

• To evaluate the performance of large language models (LLMs) like GPT-3 and Flan-T5 on RE tasks.
• To investigate the impact of varying supervision levels on generative RE approaches.
• To refine evaluation methods for generative RE by incorporating human assessments.

Main Methods:

• Utilized large language models (GPT-3, Flan-T5 large) for relation extraction as a sequence-to-sequence task.
• Employed few-shot prompting and supervised fine-tuning strategies.
• Incorporated Chain-of-Thought (CoT) explanations generated by GPT-3 for fine-tuning Flan-T5.
• Conducted human evaluations to assess performance beyond exact matching metrics.

Main Results:

• Few-shot prompting with GPT-3 achieved performance comparable to fully supervised models.
• Flan-T5, while less effective in few-shot settings, reached state-of-the-art results after fine-tuning with CoT explanations.
• Human evaluations provided a more nuanced assessment of generative RE performance.

Conclusions:

• Large language models, particularly GPT-3 with few-shot prompting, offer a powerful alternative for relation extraction.
• Chain-of-Thought fine-tuning enhances the performance of models like Flan-T5 for RE tasks.
• The study introduces a new baseline model for RE and advocates for refined evaluation methodologies.