Divide and summarize: improve SLM text summarization.

Area of Science:

• Natural Language Processing
• Artificial Intelligence

Background:

• Text summarization faces challenges, including the "Lost in the Middle" problem where models overlook information in lengthy inputs.
• Large Language Models (LLMs) and Small Language Models (SLMs) are advancing summarization, but the "Lost in the Middle" issue persists.
• The traditional "Stuff" summarization method processes text in a single pass, potentially missing crucial details.

Purpose of the Study:

• To investigate if the "Map" summarization method outperforms the "Stuff" method for texts within SLM context windows.
• To assess the effectiveness of the "Map" method in mitigating the "Lost in the Middle" problem.
• To compare the performance of SLMs using the "Map" method against LLMs using the "Stuff" method.

Main Methods:

• A two-part study involving a simulation with generated texts and automated fact-retrieval evaluation.
• A practical study focused on summarizing scientific papers.
• Comparison of "Map" and "Stuff" summarization methods using SLMs and LLMs.

Main Results:

• The "Map" method produced summaries of equal or greater accuracy compared to the "Stuff" method in both studies.
• The "Map" method demonstrated superior retention of key facts from the beginning and middle of texts.
• SLMs employing the "Map" method achieved performance comparable to LLMs using the "Stuff" method.

Conclusions:

• The "Map" method effectively addresses the "Lost in the Middle" problem in text summarization.
• For texts within SLM context windows, the "Map" method is a more effective summarization strategy than the "Stuff" method.
• The "Map" method offers a practical and efficient alternative for text summarization, particularly with SLMs.