Unsupervised cross-lingual model transfer for named entity recognition with contextualized word representations.

Area of Science:

• Natural Language Processing (NLP)
• Machine Learning
• Computational Linguistics

Background:

• Named Entity Recognition (NER) is crucial in NLP, typically requiring large annotated datasets.
• Resource-scarce languages lack sufficient annotated data for training high-performance NER models.
• Unsupervised cross-lingual transfer offers a viable solution to bridge this data gap.

Purpose of the Study:

• To investigate unsupervised cross-lingual NER using model transfer with contextualized word representations.
• To evaluate different transfer settings, including pretrained models, representation exploration, and multi-source adaptation.
• To propose and validate a novel adapter-based method combined with a parameter generation network (PGN).

Main Methods:

• Utilized transformer-based language models for contextualized word representations.
• Explored strategies for multilingual representation adaptation.
• Developed an adapter-based approach with a parameter generation network (PGN) for source-target language mapping.

Main Results:

• Achieved highly competitive performance in unsupervised cross-lingual NER through model transfer.
• Demonstrated significant improvements using the proposed adapter-based PGN model.
• Validated effectiveness on a benchmark ConLL dataset across four languages.

Conclusions:

• Unsupervised cross-lingual transfer is effective for advancing NER in low-resource scenarios.
• The proposed adapter-based PGN method offers a substantial enhancement for cross-lingual NER tasks.
• Contextualized word representations combined with effective transfer strategies are key to success.