Reaching for upper bound ROUGE score of extractive summarization methods

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Information Retrieval

Background:

• Extractive text summarization (ETS) automatically selects sentences from source text.
• The quality achievable by ETS methods needs further investigation.
• Current state-of-the-art summarization models have limitations.

Purpose of the Study:

• To evaluate the quality of summaries generated by ETS methods.
• To explore different algorithms for optimizing ETS performance.
• To compare ETS performance against existing summarization models.

Main Methods:

• Five approaches were tested to maximize ROUGE-1 scores: adapted reduced variable neighborhood search (RVNS), Greedy algorithm, VNS initialized by Greedy, genetic algorithm, and genetic algorithm initialized by Greedy.
• Experiments were conducted on articles from the arXive dataset.
• ROUGE scores (ROUGE-1 and ROUGE-2) were used as the primary evaluation metric.

Main Results:

• The genetic algorithm initialized by the Greedy algorithm results achieved the highest ROUGE-1 score of 0.59 and ROUGE-2 score of 0.25.
• These scores are superior to the current state-of-the-art on the same dataset, which reported a ROUGE-1 score of 0.46.
• The tested approaches demonstrate the potential of ETS methods.

Conclusions:

• ETS methods, particularly when enhanced with advanced algorithms like the combined genetic and greedy approach, can produce highly effective text summaries.
• There is significant potential for further development and application of ETS techniques.
• ETS methods are a valuable and currently underutilized area in automatic text summarization.