A two-stage fine-tuning method for low-resource cross-lingual summarization.

Area of Science:

• Natural Language Processing
• Machine Learning
• Artificial Intelligence

Background:

• Cross-lingual summarization (CLS) requires understanding semantics across languages and compressing information.
• Existing methods include pipeline (translate-then-summarize) and end-to-end approaches using multilingual pre-trained models (mPTMs).
• mPTMs often struggle with semantic alignment for low-resource languages due to training on resource-rich data.

Purpose of the Study:

• To propose a novel two-stage fine-tuning method (TFLCLS) for low-resource cross-lingual summarization.
• To enhance semantic alignment and information compression capabilities of mPTMs for low-resource languages.
• To address the limitations of current CLS methods in low-resource scenarios.

Main Methods:

• Developed a two-stage fine-tuning approach: semantic alignment fine-tuning followed by adaptive joint fine-tuning.
• Stage 1 focuses on improving mPTMs' understanding of low-resource languages.
• Stage 2 enhances both semantic alignment and information compression for CLS tasks.

Main Results:

• Introduced three new low-resource CLS datasets: Vi2ZhLow, En2ZhLow, and Zh2EnLow.
• TFLCLS significantly outperformed state-of-the-art methods, achieving improvements of 18.88%, 12.71%, and 16.91% in ROUGE-2.
• Effective performance was demonstrated even with limited training data (5,000 samples).

Conclusions:

• The proposed TFLCLS method effectively addresses the challenges of low-resource cross-lingual summarization.
• The two-stage fine-tuning strategy enhances crucial semantic alignment and information compression abilities.
• TFLCLS offers a promising solution for improving CLS performance in data-scarce language pairs.