Generating EDU Extracts for Plan-Guided Summary Re-Ranking.

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Computational Linguistics

Background:

• Two-step summarization (candidate generation then re-ranking) improves ROUGE scores over single-step methods.
• Standard decoding methods often produce redundant and low-quality summary candidates.

Purpose of the Study:

• To develop a novel method for generating high-quality summary candidates for re-ranking.
• To address redundancy and quality issues in standard decoding methods for abstractive summarization.

Main Methods:

• Generated unique content plans for each abstract using a BART language model (LM) with an extractive copy mechanism.
• Produced distinct plan-guided abstractive candidates using the top beams from the content plan generator.
• Applied an existing re-ranker (BRIO) to evaluate generated candidates against baseline methods.

Main Results:

• Achieved significant relevance improvements on CNN/Dailymail, NYT, and Xsum corpora, with ROUGE-2 F1 gains of 0.88, 2.01, and 0.38, respectively.
• Human evaluation on CNN/DM validated the superiority of the proposed method.
• Prompting GPT-3 with elemental discourse unit (EDU) plans outperformed sampling methods by 1.05 ROUGE-2 F1 points on 1k CNN/DM samples.

Conclusions:

• The proposed content plan-guided generation method effectively produces high-quality, distinct summary candidates.
• This approach offers a significant advancement over standard decoding techniques in abstractive summarization.
• The method demonstrates strong performance across multiple datasets and evaluation types, including human assessment.