Deep contextualized embeddings for quantifying the informative content in biomedical text summarization.

Area of Science:

• Natural Language Processing
• Biomedical Informatics
• Artificial Intelligence

Background:

• Biomedical text summarization faces challenges in capturing contextual information.
• Deep bidirectional language models offer advanced capabilities for understanding text context.

Purpose of the Study:

• To demonstrate the utility of contextualized embeddings from deep bidirectional language models for quantifying sentence informativeness in biomedical text summarization.
• To develop and evaluate a novel summarization method leveraging these embeddings.

Main Methods:

• Proposed a novel summarization method using contextualized embeddings from Bidirectional Encoder Representations from Transformers (BERT).
• Combined different BERT versions with a clustering approach to identify key sentences.
• Evaluated the summarizer's performance using the ROUGE toolkit against existing methods.

Main Results:

• The proposed summarizer achieved state-of-the-art results, significantly outperforming existing domain-specific and domain-independent methods.
• Larger BERT models, even without specific biomedical pretraining, showed superior performance.
• Among models of similar size, those with biomedical pretraining yielded the best outcomes.

Conclusions:

• A hybrid system integrating deep bidirectional language models and clustering achieves state-of-the-art performance.
• This approach bypasses the need for manual feature creation or computationally intensive domain-specific pretraining.
• The study establishes a foundation for future research into deep contextualized language models for biomedical text summarization.