xMEN: a modular toolkit for cross-lingual medical entity normalization.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Bioinformatics

Background:

• Medical entity normalization is crucial for information extraction in healthcare.
• Existing systems often struggle with low-resource languages due to limited terminology and annotations.
• Improving cross-lingual capabilities is essential for global health informatics.

Purpose of the Study:

• To develop a robust system for cross-lingual medical entity normalization (MEN) that performs well in both high- and low-resource scenarios.
• To address the scarcity of language-specific resources by utilizing multilingual information.
• To introduce novel methods for training and re-ranking medical entities in a cross-lingual context.

Main Methods:

• Proposed xMEN, a modular system for cross-lingual MEN.
• Employed cross-lingual candidate generation using multilingual aliases.
• Incorporated a trainable cross-encoder (CE) for candidate ranking, with a novel rank regularization term.
• Developed weakly labeled datasets for re-ranking in low-resource scenarios using machine translation and annotation projection.

Main Results:

• xMEN achieved state-of-the-art performance on benchmark datasets for several European languages.
• Demonstrated the effectiveness of weakly supervised CEs in scenarios lacking target task training data.
• Identified complex entities as a remaining challenge for normalization.

Conclusions:

• xMEN provides strong performance for medical entity normalization across diverse languages, even with limited labeled data and terminology.
• The system's modular design allows for easy integration of new modules and datasets.
• An open-source Python toolkit is released to facilitate reproducible research and future benchmarks.