A domain-specific cross-lingual semantic alignment learning model for low-resource languages

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Machine Learning

Background:

• Cross-lingual semantic alignment models are crucial for multilingual domain-specific data utilization.
• Existing methods struggle with parallel data scarcity, semantic heterogeneity, and morphological complexity, particularly in agglutinative languages.

Purpose of the Study:

• Propose CLWKD, a novel cross-lingual mapping and knowledge distillation framework.
• Enhance cross-lingual knowledge transfer for low-resource languages using domain-specific pretrained models.
• Address challenges in data scarcity and morphological complexity for agglutinative languages.

Main Methods:

• CLWKD integrates multi-granularity alignment matrices (token, word, sentence) with limited parallel data.
• Employs multilingual embedding sharing and morphological segmentation for agglutinative languages.
• Utilizes generator pretraining and parameter recycling for stable, efficient mapping.

Main Results:

• CLWKD demonstrates effectiveness across medical, legal, and educational domains.
• Successful cross-lingual alignment achieved for Mongolian-Chinese and Korean-Chinese language pairs.
• Improved performance in three cross-lingual tasks, addressing data scarcity and structural differences.

Conclusions:

• CLWKD offers a robust solution for cross-lingual semantic alignment, particularly for morphologically rich languages.
• The framework effectively leverages knowledge distillation and multi-granularity alignment.
• Facilitates cost-effective data sharing and improves low-resource language task performance.