Abstractive summarization through the prism of decoding strategies.

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Machine Learning

Background:

• Transformer-based language models (LMs) have accelerated natural language generation (NLG), particularly in abstractive summarization (AS).
• Decoding strategies, crucial for summary quality, are often overlooked despite numerous heuristics and hyperparameters.
• A lack of guidance exists for selecting optimal decoding techniques based on specific tasks and metrics.

Purpose of the Study:

• To conduct a comprehensive comparative assessment of decoding-time techniques for abstractive summarization.
• To provide guidance for informed decision-making regarding decoding strategies in AS.
• To evaluate the effectiveness and efficiency of various decoding options across different AS tasks and LM architectures.

Main Methods:

• Explored over 3,500 combinations of decoding settings with large-scale autoregressive and decoder-only LMs across six datasets.
• Performed comparative assessment of decoding strategies for short, long, and multi-document AS.
• Quantitatively measured performance using ten automatic metrics (semantic similarity, factuality, compression, redundancy, carbon footprint) and human evaluation.

Main Results:

• Optimized decoding choices lead to substantial improvements in abstractive summarization performance.
• Significant variations in effectiveness and efficiency were observed across different decoding strategies and tasks.
• The study identified key decoding parameters influencing summary quality and resource utilization.

Conclusions:

• Effective decoding strategy selection is critical for maximizing the performance of abstractive summarization models.
• The findings offer valuable insights for researchers and practitioners in NLG.
• Introduced Prism, a novel dataset to facilitate differentiable optimization of decoding options for AS.