Coreference Resolution Based on High-Dimensional Multi-Scale Information

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Artificial Intelligence

Background:

• Coreference resolution is a fundamental Natural Language Processing (NLP) task crucial for text understanding.
• Evaluating similarity in long-span texts presents challenges for traditional text-level encoding methods.
• Existing models struggle to effectively capture global context in extended documents.

Purpose of the Study:

• To investigate methods for enhancing global information collection in BERT encoding for NLP tasks.
• To design a novel module that improves BERT's applicability across various text spans.
• To address the challenge of evaluating long-span text similarity in coreference resolution.

Main Methods:

• Comparative analysis of methods to improve BERT's global information collection.
• Development of a multi-scale context information module tailored for different text spans.
• Application of dimension expansion to enhance linear separability.
• Utilizing cross-entropy loss as the optimization objective.

Main Results:

• The proposed multi-scale context information module was integrated with BERT and span BERT.
• BERT encoding performance saw an improvement of 0.5% in F1 score.
• Span BERT encoding performance improved by 0.2% in F1 score.
• The module demonstrated enhanced ability to handle long-span text contexts.

Conclusions:

• The developed multi-scale context information module effectively improves BERT's performance on coreference resolution tasks, particularly for long texts.
• The approach enhances the model's capacity to capture global information and handle varying text spans.
• Future work could explore further optimizations and applications of this module in other NLP tasks.